SkillCurated by metahubEval warningsprompts

dataset-profiler

Name: dataset-profiler
Author: oxbshw

Use when first encountering a new dataset — produces a structured profile (schema, missingness, distributions, outliers, gotchas) before any analysis.

pinned to #7f8ee3cupdated 2 months ago

7/11Verified publisherCurated by MetaHubActively maintainedLicensedDocumentedStructuredClean repo

Ask your AI client: “install skills/dataset-profiler”.

Requires the metahub MCP server installed in your client. Set up MCP.

mh install skills/dataset-profiler

metahub onboarded this repo on the author's behalf.

If you own github.com/oxbshw/LLM-Agents-Ecosystem-Handbook on GitHub, claim the listing to take over publishing. Your claim preserves the existing eval history and badges; only the curator label is replaced with verified-publisher on your next publish.

Claim this repo

Stars

533

Last commit

2 months ago

7f8ee3c

Latest release

published

2 months ago

#ai
#ai-agent
#ai-agents
#fine-tuning
#finetuning-llms
#freamework
#llm
#llmops
#local-development
#mcp-server
#memory
#rag
#rag-chatbot
#voice-agent

Evaluation report

WarningsView full report

Automated checks the publisher passed at publish time — structure, docs, safety, and whether the artifact behaves as claimed.7f8ee3c· 2 months ago

Behavioral checks ran but aren't published for this artifact; the static checks above ran at publish time.

Documentation

4 passed1 warning

README has usage / example sectionswarn
README has no Usage / Example / Quick start / Installation heading at any level
Add a `# Usage`, `## Quick start`, or similar section so end users can copy/paste a working invocation. Code blocks alone work too if there are several.
Description quality
20 words · 150 chars — "Use when first encountering a new dataset — produces a structured profile (schem…"
README is present and substantial
19,199 chars · 20 sections · 2 code blocks
Tags / topics declared
14 total — ai, ai-agent, ai-agents, fine-tuning, finetuning-llms, freamework (+8)
Homepage / docs URL declared
no homepage declared (registry will use the repo URL) — info-only, not blocking

Release history

releasecurrent7f8ee3cwarn2 months ago

View on GitHub

When to use

A new dataset arrives and you need to understand it before using it
Before reproducing an analysis that referenced a dataset
When data quality is suspect ("the chart looked wrong")

When NOT to use

Streaming / online data (this is point-in-time)
Sensitive PII without an explicit allow-list

Inputs

Name	Type	Required	Notes
`path`	path	yes	CSV / Parquet / JSONL
`target`	string	no	column of interest (gets extra distribution detail)

Outputs

profile.md with: Source, Schema, Missingness, Distributions, Outliers, Joins / keys, Gotchas, Open questions.

Workflow

Load with the right reader (extension-detected); record row count, file size
Schema: column → dtype → nullable → example value
Missingness: % per column, top columns by missingness
Distributions: numeric (min, p50, p95, max, std), categorical (top-k, cardinality)
Outliers: flag rows beyond p99 + 3·IQR for numerics
Identify potential keys (unique columns) and join candidates
Gotchas: timezone columns, mixed encodings, suspicious all-zero rows, magic values (-1, 9999-12-31)
Open questions: ambiguous columns / values that need owner input

References

references/profile-template.md

Success criteria

Every column appears in Schema + Missingness
Outliers section includes example rows
Gotchas section is non-empty (real datasets always have some)

Failure modes

File too large to read in memory → switch to streaming + sampled stats; flag prominently
Encoding fails → try common alternatives; if all fail, surface and stop