jeffallan/claude-skills

pandas-pro

Performs pandas DataFrame operations for data analysis, manipulation, and transformation.

View source

How to use

  1. Copy the installation command and review the source package before installing it in your own agent environment.
  2. Check the source document for required tools, credentials and supported hosts.
  3. Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/jeffallan/claude-skills --skill pandas-pro

Before you start

  • Installation does not include a model subscription, API allowance or permission to access your files.
  • Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

Pandas Pro

Expert pandas developer specializing in efficient data manipulation, analysis, and transformation workflows with production-grade performance patterns.

Core Workflow

  1. Assess data structure — Examine dtypes, memory usage, missing values, data quality:
python
   print(df.dtypes)
   print(df.memory_usage(deep=True).sum() / 1e6, "MB")
   print(df.isna().sum())
   print(df.describe(include="all"))
  1. Design transformation — Plan vectorized operations, avoid loops, identify indexing strategy
  2. Implement efficiently — Use vectorized methods, method chaining, proper indexing
  3. Validate results — Check dtypes, shapes, null counts, and row counts:
python
   assert result.shape[0] == expected_rows, f"Row count mismatch: {result.shape[0]}"
   assert result.isna().sum().sum() == 0, "Unexpected nulls after transform"
   assert set(result.columns) == expected_cols
  1. Optimize — Profile memory, apply categorical types, use chunking if needed

Reference Guide

Load detailed guidance based on context:

TopicReferenceLoad When
DataFrame Operationsreferences/dataframe-operations.mdIndexing, selection, filtering, sorting
Data Cleaningreferences/data-cleaning.mdMissing values, duplicates, type conversion
Aggregation & GroupByreferences/aggregation-groupby.mdGroupBy, pivot, crosstab, aggregation
Merging & Joiningreferences/merging-joining.mdMerge, join, concat, combine strategies
Performance Optimizationreferences/performance-optimization.mdMemory usage, vectorization, chunking

Code Patterns

Vectorized Operations (before/after)

python
# ❌ AVOID: row-by-row iteration
for i, row in df.iterrows():
    df.at[i, 'tax'] = row['price'] * 0.2

# ✅ USE: vectorized assignment
df['tax'] = df['price'] * 0.2

Safe Subsetting with .copy()

python
# ❌ AVOID: chained indexing triggers SettingWithCopyWarning
df['A']['B'] = 1

# ✅ USE: .loc[] with explicit copy when mutating a subset
subset = df.loc[df['status'] == 'active', :].copy()
subset['score'] = subset['score'].fillna(0)

GroupBy Aggregation

python
summary = (
    df.groupby(['region', 'category'], observed=True)
    .agg(
        total_sales=('revenue', 'sum'),
        avg_price=('price', 'mean'),
        order_count=('order_id', 'nunique'),
    )
    .reset_index()
)

Merge with Validation

python
merged = pd.merge(
    left_df, right_df,
    on=['customer_id', 'date'],
    how='left',
    validate='m:1',          # asserts right key is unique
    indicator=True,
)
unmatched = merged[merged['_merge'] != 'both']
print(f"Unmatched rows: {len(unmatched)}")
merged.drop(columns=['_merge'], inplace=True)

Missing Value Handling

python
# Forward-fill then interpolate numeric gaps
df['price'] = df['price'].ffill().interpolate(method='linear')

# Fill categoricals with mode, numerics with median
for col in df.select_dtypes(include='object'):
    df[col] = df[col].fillna(df[col].mode()[0])
for col in df.select_dtypes(include='number'):
    df[col] = df[col].fillna(df[col].median())

Time Series Resampling

python
daily = (
    df.set_index('timestamp')
    .resample('D')
    .agg({'revenue': 'sum', 'sessions': 'count'})
    .fillna(0)
)

Pivot Table

python
pivot = df.pivot_table(
    values='revenue',
    index='region',
    columns='product_line',
    aggfunc='sum',
    fill_value=0,
    margins=True,
)

Memory Optimization

python
# Downcast numerics and convert low-cardinality strings to categorical
df['category'] = df['category'].astype('category')
df['count'] = pd.to_numeric(df['count'], downcast='integer')
df['score'] = pd.to_numeric(df['score'], downcast='float')
print(df.memory_usage(deep=True).sum() / 1e6, "MB after optimization")

Constraints

MUST DO

  • Use vectorized operations instead of loops
  • Set appropriate dtypes (categorical for low-cardinality strings)
  • Check memory usage with .memory_usage(deep=True)
  • Handle missing values explicitly (don't silently drop)
  • Use method chaining for readability
  • Preserve index integrity through operations
  • Validate data quality before and after transformations
  • Use .copy() when modifying subsets to avoid SettingWithCopyWarning

MUST NOT DO

  • Iterate over DataFrame rows with .iterrows() unless absolutely necessary
  • Use chained indexing (df['A']['B']) — use .loc[] or .iloc[]
  • Ignore SettingWithCopyWarning messages
  • Load entire large datasets without chunking
  • Use deprecated methods (.ix, .append() — use pd.concat())
  • Convert to Python lists for operations possible in pandas
  • Assume data is clean without validation

Output Templates

When implementing pandas solutions, provide:

  1. Code with vectorized operations and proper indexing
  2. Comments explaining complex transformations
  3. Memory/performance considerations if dataset is large
  4. Data validation checks (dtypes, nulls, shapes)

Maintained by @jeffallan, Principal Consultant at Synergetic Solutions

Documentation

Related by task

from this repository

More skills

All skills
jeffallan
Community

typescript-pro

Implements advanced TypeScript type systems, creates custom type guards, utility types, and branded types, and configures tRPC for end-to-end type safety. Use when building TypeScript applications requiring advanced generics, conditional or mapped types, discriminated unions, monorepo setup, or full-stack type safety with tRPC.

1h installs
2
GitHub stars
12.5K
Updated
Nov 20
jeffallan
Community

rust-engineer

Writes, reviews, and debugs idiomatic Rust code with memory safety and zero-cost abstractions. Implements ownership patterns, manages lifetimes, designs trait hierarchies, builds async applications with tokio, and structures error handling with Result/Option. Use when building Rust applications, solving ownership or borrowing issues, designing trait-based APIs, implementing async/await concurrency, creating FFI bindings, or optimizing for performance and memory safety. Invoke for Rust, Cargo, ow

1h installs
1
GitHub stars
12.5K
Updated
Nov 20
jeffallan
Community

flutter-expert

Use when building cross-platform applications with Flutter 3+ and Dart. Invoke for widget development, Riverpod/Bloc state management, GoRouter navigation, platform-specific implementations, performance optimization.

1h installs
1
GitHub stars
12.5K
Updated
Nov 16
jeffallan
Community

javascript-pro

Writes, debugs, and refactors JavaScript code using modern ES2023+ features, async/await patterns, ESM module systems, and Node.js APIs. Use when building vanilla JavaScript applications, implementing Promise-based async flows, optimising browser or Node.js performance, working with Web Workers or Fetch API, or reviewing .js/.mjs/.cjs files for correctness and best practices.

1h installs
1
GitHub stars
12.5K
Updated
Nov 16