Mode d’emploi
- Copiez la commande et examinez le paquet avant de l’installer dans votre environnement agent.
- Consultez le document original pour les outils, identifiants et environnements requis.
- Décrivez les entrées et le livrable souhaité ; comparez la sortie aux sources.
npx skills add https://github.com/majesticlabs-dev/majestic-marketplace --skill great-expectationsAvant de commencer
- L’installation n’inclut ni abonnement au modèle, ni quota API, ni accès aux fichiers.
- Étoiles et installations reflètent l’activité, pas un test indépendant de qualité ou de sécurité.
Document Skill original
Rendu depuis le dépôt source en conservant titres, exemples, code, tableaux, liens et images.
Great Expectations
Audience: Data engineers building validated data pipelines.
Goal: Provide GX patterns for expectation-based validation and monitoring.
Scripts
Execute GX functions from scripts/expectations.py:
python
from scripts.expectations import (
get_pandas_context,
add_dataframe_asset,
create_basic_suite,
run_validation
)Usage Examples
Quick Setup
python
from scripts.expectations import get_pandas_context, add_dataframe_asset
context, datasource = get_pandas_context("my_datasource")
batch_request = add_dataframe_asset(datasource, "users", df)Create Expectation Suite
python
from scripts.expectations import create_basic_suite
columns_config = {
'user_id': {'not_null': True, 'unique': True, 'type': 'int'},
'age': {'min': 0, 'max': 150},
'status': {'values': ['active', 'inactive', 'pending']},
'email': {'regex': r'^[\w\.-]+@[\w\.-]+\.\w+$'}
}
suite = create_basic_suite(context, "user_suite", columns_config)Run Validation
python
from scripts.expectations import run_validation
results = run_validation(
context,
checkpoint_name="user_checkpoint",
batch_request=batch_request,
suite_name="user_suite"
)
if results['success']:
print("All expectations passed!")
else:
for failure in results['failures']:
print(f"Failed: {failure['expectation']} on {failure['column']}")Common Expectations Reference
| Category | Expectation | Description |
|---|---|---|
| Table | ExpectTableRowCountToBeBetween | Row count range |
| Existence | ExpectColumnToExist | Column must exist |
| Nulls | ExpectColumnValuesToNotBeNull | No null values |
| Range | ExpectColumnValuesToBeBetween | Value bounds |
| Set | ExpectColumnValuesToBeInSet | Allowed values |
| Pattern | ExpectColumnValuesToMatchRegex | Regex match |
| Unique | ExpectColumnValuesToBeUnique | No duplicates |
Data Docs
python
# Build and open HTML reports
context.build_data_docs()
context.open_data_docs()Directory Structure
great_expectations/
├── great_expectations.yml # Config
├── expectations/ # Expectation suites (JSON)
├── checkpoints/ # Checkpoint definitions
├── plugins/ # Custom expectations
└── uncommitted/
├── data_docs/ # Generated HTML docs
└── validations/ # Validation resultsWhen to Use Great Expectations
| Use Case | GX | Alternative |
|---|---|---|
| Pipeline monitoring | ✓ | - |
| Data warehouse validation | ✓ | - |
| Automated data docs | ✓ | - |
| Simple DataFrame checks | - | Pandera |
| Record-level API validation | - | Pydantic |
Dependencies
great_expectations>=0.18
pandas
