Add card ui_schema validation guard and fix drift (#199)

CHANGELOG.md

@@ -8,6 +8,19 @@ Contributors add user-facing entries under `[Unreleased]` in the same PR. Mainta
 
 ## [Unreleased]
 
+### Added
+
+- **Framework:** `skillware/core/ui_schema.py` helpers to resolve dot paths and validate output-card field keys (#199).
+- **CI:** Parametrized guard `tests/test_card_ui_schema.py` — every registry skill with output-card `ui_schema` must ship a fixture under `tests/fixtures/card_ui_schema/` whose samples cover all field keys (#199).
+
+### Fixed
+
+- **`compliance/mica_module`**, **`compliance/pii_masker`**: Align `card.json` output fields with actual `execute()` return shape (#199).
+
+### Changed
+
+- **Documentation:** CONTRIBUTING and contributor workflow checklists — update `card.json` and output fixtures together when changing `execute()` output (#199).
+
 ## [0.4.5] - 2026-07-16
 
 ### Added

CONTRIBUTING.md

@@ -269,6 +269,7 @@ requirements:
 - Catch internal errors and return a structured error report; do not crash the host agent.
 - Do **not** print to stdout or stderr for normal operation.
 - Do **not** embed open-ended LLM code generation as the skill implementation.
+- When you change the JSON shape returned by `execute()`, update `card.json` output fields (if present) and the matching fixture under `tests/fixtures/card_ui_schema/` in the same PR.
 
 ### 3. `instructions.md` (cognition)
 
@@ -284,6 +285,8 @@ The primary guide for the host LLM.
 - Optional but recommended for user-facing agents and catalog UIs.
 - Describes UI presentation (`name`, `description`, `icon`, `ui_schema`, and similar).
 - When present, include an `issuer` object that matches `manifest.yaml` (`name` and `email` at minimum; copy `github` and `org` when used).
+- For output cards (`ui_schema.type` = `card`), each `ui_schema.fields[].key` must be a dot path into the JSON returned by `execute()` (for example `metadata.wallet_address`, `preview.you_pay`). Update `card.json` in the same PR when you change the output shape.
+- Add or refresh a representative output fixture at `tests/fixtures/card_ui_schema/<category>__<skill_name>.json` (one object or a `{"samples": [...]}` list when multiple execute paths surface different fields). CI validates keys via `tests/test_card_ui_schema.py` (#199).
 
 ### 5. `test_skill.py` (bundle test)
 

docs/TESTING.md

@@ -19,6 +19,7 @@ Tests fall into four layers: **bundle**, **framework**, **maintainer**, and **ex
 | GitHub label policy test (`test_github_labels.py`) | Done |
 | PyPI wheel packaging smoke test (`scripts/wheel_smoke_test.py`) | Done |
 | Optional extras sync (`scripts/sync_extras.py`, `tests/test_extras_sync.py`) | Done |
+| Card UI schema vs execute output (`tests/test_card_ui_schema.py`) | Done |
 
 Every pull request runs `black --check`, `flake8`, `pytest skills/`, `pytest tests/`, and a **wheel-smoke** job that builds a wheel, installs it in a fresh venv (base install only — no `[all]` or per-skill extras), and verifies every bundled registry skill is present and loadable. Bundle tests gate merge the same as framework and maintainer tests.
 
@@ -62,6 +63,7 @@ pip install -r requirements.txt
 
 - Core engine health: loader, CLI, issuer rules, version policy, parameter schema validation (`tests/test_validate_params.py`).
 - `tests/test_skill_issuer.py` also enforces registry packaging (`__init__.py`), issuer metadata, presence of `test_skill.py` in every skill bundle, and rejects legacy manifest `output:` keys.
+- `tests/test_card_ui_schema.py` validates output-card `ui_schema.fields[].key` dot paths against fixtures in `tests/fixtures/card_ui_schema/` (#199).
 - `tests/test_registry_docs.py` enforces doc-drift parity: skill catalog index matches manifests, examples README matches scripts on disk, and agent-loops.md references every registered skill.
 - Lives at the **root of `tests/`** only (`tests/test_loader.py`, `tests/test_cli.py`, …).
 - Clone-repo only; runs in CI via `pytest tests/` together with maintainer tests below.
@@ -84,7 +86,7 @@ pip install -r requirements.txt
 | :--- | :--- | :--- |
 | Manifest + execute contract for one skill | Bundle test | `skills/compliance/tos_evaluator/test_skill.py` |
 | Loader path + mocked externals (optional depth) | Maintainer test | `tests/skills/compliance/test_tos_evaluator.py` |
-| Loader, CLI, registry issuer rules, param validation | Framework test | `tests/test_loader.py`, `tests/test_skill_issuer.py`, `tests/test_validate_params.py`, `tests/test_registry_docs.py` |
+| Loader, CLI, registry issuer rules, param validation | Framework test | `tests/test_loader.py`, `tests/test_skill_issuer.py`, `tests/test_validate_params.py`, `tests/test_registry_docs.py`, `tests/test_card_ui_schema.py` |
 | End-to-end provider demo script | Usage example | `examples/gemini_tos_evaluator.py` |
 
 **Rule of thumb:** if it ships with the skill and must pass before merge → **bundle test** (CI + local). If it is extra regression depth for clone-repo work → **maintainer test** (optional). If it proves provider integration → **example**, not pytest.

docs/contributing/ai_native_workflow.md

@@ -239,6 +239,7 @@ These align with [CONTRIBUTING.md](../../CONTRIBUTING.md). Violations block merg
 - `manifest.yaml` `name` must equal `category/skill_name` (matches folder path); loader warns on mismatch for registry layout
 - `issuer.name` and `issuer.email` required; `github` and `org` optional; no template placeholders in registry paths
 - `card.json` issuer must match manifest `name` and `email` when present
+- Output-card `ui_schema.fields[].key` values must resolve in `execute()` JSON; keep `tests/fixtures/card_ui_schema/<category>__<skill_name>.json` in sync (#199)
 - Update `docs/skills/<skill_name>.md` and `docs/skills/README.md`
 - On each catalog page, add a **Usage Examples** section (Gemini, Claude, OpenAI, DeepSeek, Ollama prompt mode) per [skill usage template](../usage/skill_usage_template.md). Keep provider mechanics in `docs/usage/`; put skill-specific paths, sample user messages, and `execute` payloads on the skill page.
 - Categories: `compliance`, `creative`, `data_engineering`, `defi`, `dev_tools`, `finance`, `monitoring`, `office`, `optimization`, `wellness` — see [Skill library](../skills/README.md) for the live registry; [Choosing a category](../../CONTRIBUTING.md#choosing-a-category) in CONTRIBUTING.md (issue first for new top-level folders)
@@ -276,7 +277,7 @@ Complete the checklist that matches your issue during Stage 5.
 - [ ] Optional: `short_description` field (~80 chars) for a concise one-line summary in `skillware list`
 - [ ] `skill.py`: exactly one `BaseSkill` subclass (auto-discovered as `bundle["class"]`); deterministic, JSON-serializable returns, safe error handling
 - [ ] `instructions.md`: when to use, how to interpret output, limitations
-- [ ] `card.json`: `issuer` matches manifest
+- [ ] `card.json`: `issuer` matches manifest; output-card `ui_schema.fields[].key` paths resolve in `tests/fixtures/card_ui_schema/<category>__<skill_name>.json` (update fixture when `execute()` output changes)
 - [ ] `test_skill.py` (bundle test) passes — `pytest skills/<category>/<skill_name>/test_skill.py` or `skillware test <category>/<skill_name>`
 - [ ] Bundle tests mock all network calls and model downloads; CI does not download models.
 - [ ] `docs/skills/<skill_name>.md` and catalog row in `docs/skills/README.md` (include **Recommended install:** `pip install "skillware[<category>_<skill>]"` per [install_extras.md](../usage/install_extras.md))

skills/compliance/mica_module/card.json

@@ -13,12 +13,18 @@
         "type": "card",
         "fields": [
             {
-                "key": "run_evaluator",
-                "label": "Enable Evaluator"
+                "key": "policy_status",
+                "label": "Policy Status",
+                "type": "status_badge"
             },
             {
-                "key": "evaluator_model",
-                "label": "Evaluator Model"
+                "key": "gemini_evaluator_feedback.grade",
+                "label": "Evaluator Grade"
+            },
+            {
+                "key": "retrieved_sections",
+                "label": "Sections",
+                "type": "tags"
             }
         ]
     }

skills/compliance/pii_masker/card.json

@@ -13,12 +13,18 @@
         "type": "card",
         "fields": [
             {
-                "key": "mode",
-                "label": "Redaction Mode"
+                "key": "metadata.entity_count",
+                "label": "Entities Found",
+                "type": "count"
             },
             {
-                "key": "ollama_url",
-                "label": "Ollama Endpoint"
+                "key": "metadata.detected_entities",
+                "label": "Entity Types",
+                "type": "tags"
+            },
+            {
+                "key": "metadata.security_level",
+                "label": "Security Level"
             }
         ]
     }

skillware/core/ui_schema.py

@@ -0,0 +1,99 @@
+"""Helpers for validating card.json ui_schema field keys against skill output."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Iterable, List, Mapping, Sequence, Union
+
+JsonObject = Dict[str, Any]
+
+
+def resolve_dot_path(data: Mapping[str, Any], path: str) -> Any:
+    """Resolve a dot-separated path against a nested mapping.
+
+    Raises:
+        KeyError: if any segment is missing.
+        TypeError: if a segment is not a mapping when further segments remain.
+    """
+    if not path or not path.strip():
+        raise KeyError("empty path")
+
+    current: Any = data
+    for segment in path.split("."):
+        if not isinstance(current, Mapping):
+            raise KeyError(path)
+        if segment not in current:
+            raise KeyError(path)
+        current = current[segment]
+    return current
+
+
+def path_exists(data: Mapping[str, Any], path: str) -> bool:
+    """Return True when ``path`` resolves in ``data``."""
+    try:
+        resolve_dot_path(data, path)
+    except (KeyError, TypeError):
+        return False
+    return True
+
+
+def is_output_card_ui_schema(ui_schema: Any) -> bool:
+    """True when ui_schema declares output card fields (type=card + fields list)."""
+    if not isinstance(ui_schema, Mapping):
+        return False
+    if ui_schema.get("type") != "card":
+        return False
+    fields = ui_schema.get("fields")
+    return isinstance(fields, list) and bool(fields)
+
+
+def extract_card_field_keys(ui_schema: Mapping[str, Any]) -> List[str]:
+    """Return ui_schema.fields[].key values for output card schemas."""
+    if not is_output_card_ui_schema(ui_schema):
+        return []
+
+    keys: List[str] = []
+    for field in ui_schema.get("fields", []):
+        if not isinstance(field, Mapping):
+            continue
+        key = field.get("key")
+        if isinstance(key, str) and key.strip():
+            keys.append(key.strip())
+    return keys
+
+
+def missing_keys_for_samples(
+    samples: Sequence[Mapping[str, Any]], keys: Iterable[str]
+) -> List[str]:
+    """Return field keys that do not resolve in any sample output."""
+    key_list = list(keys)
+    missing: List[str] = []
+    for key in key_list:
+        if not any(path_exists(sample, key) for sample in samples):
+            missing.append(key)
+    return missing
+
+
+def normalize_fixture_samples(
+    payload: Union[JsonObject, Sequence[JsonObject]],
+) -> List[JsonObject]:
+    """Accept a single output object or {"samples": [...]} fixture payload."""
+    if isinstance(payload, Mapping) and "samples" in payload:
+        raw_samples = payload.get("samples")
+        if not isinstance(raw_samples, list) or not raw_samples:
+            raise ValueError("fixture samples must be a non-empty list")
+        return [sample for sample in raw_samples if isinstance(sample, Mapping)]
+
+    if isinstance(payload, Mapping):
+        return [payload]
+
+    raise ValueError("fixture must be an object or a samples list wrapper")
+
+
+def validate_card_ui_schema(
+    ui_schema: Mapping[str, Any], samples: Sequence[Mapping[str, Any]]
+) -> List[str]:
+    """Validate output card field keys against one or more execute() samples."""
+    keys = extract_card_field_keys(ui_schema)
+    if not keys:
+        return []
+    return missing_keys_for_samples(samples, keys)

templates/python_skill/card.json

@@ -14,7 +14,8 @@
         "fields": [
             {
                 "key": "status",
-                "label": "Status"
+                "label": "Status",
+                "type": "status_badge"
             }
         ]
     }

tests/fixtures/card_ui_schema/compliance__mica_module.json

@@ -0,0 +1,14 @@
+{
+  "samples": [
+    {
+      "retrieved_sections": ["Title V / Article 59"],
+      "policy_status": "CAUTION",
+      "gemini_evaluator_feedback": {
+        "grade": "N/A",
+        "holes_found": "Evaluator disabled by parameter.",
+        "suggestion": "Proceed manually integrating the extracted logic."
+      },
+      "final_context_for_agent": "Output your final answer seamlessly integrating MiCA rules."
+    }
+  ]
+}

tests/fixtures/card_ui_schema/compliance__pii_masker.json

@@ -0,0 +1,13 @@
+{
+  "samples": [
+    {
+      "sanitized_text": "Contact [EMAIL_1] for support.",
+      "metadata": {
+        "detected_entities": ["EMAIL"],
+        "entity_count": 1,
+        "security_level": "local-only",
+        "model": "arpacorp/micro-f1-mask"
+      }
+    }
+  ]
+}

tests/fixtures/card_ui_schema/compliance__tos_evaluator.json

@@ -0,0 +1,27 @@
+{
+  "samples": [
+    {
+      "is_safe_to_proceed": true,
+      "confidence_score": 0.72,
+      "verdict": "SAFE",
+      "reason": "Discovered policy text includes language that appears permissive for the requested behavior.",
+      "recommended_next_step": "Proceed conservatively and continue honoring crawl delays and rate limits.",
+      "action_type": "read",
+      "robots_assessment": {
+        "can_fetch": true,
+        "reason": "Allowed by robots.txt"
+      },
+      "tos_assessment": {
+        "status": "allowed",
+        "summary": "Permissive language found."
+      },
+      "llm_assessment": {
+        "status": "not_used"
+      },
+      "discovered_policy_urls": {
+        "candidates": ["https://example.com/terms"]
+      },
+      "evidence": []
+    }
+  ]
+}

tests/fixtures/card_ui_schema/creative__bg_remover.json

@@ -0,0 +1,13 @@
+{
+  "samples": [
+    {
+      "success": true,
+      "image_base64": "iVBORw0KGgo=",
+      "mime_type": "image/png",
+      "output_path": "/tmp/bg_removed.png",
+      "width": 640,
+      "height": 480,
+      "model_used": "u2net"
+    }
+  ]
+}

tests/fixtures/card_ui_schema/data_engineering__synthetic_generator.json

@@ -0,0 +1,11 @@
+{
+  "samples": [
+    {
+      "samples": [{"id": 1, "value": "sample-a"}],
+      "entropy_score": 4.2,
+      "status": "success",
+      "provider_used": "gemini",
+      "samples_generated": 1
+    }
+  ]
+}

tests/fixtures/card_ui_schema/defi__evm_tx_handler.json

@@ -0,0 +1,20 @@
+{
+  "samples": [
+    {
+      "status": "ready",
+      "preview": {
+        "you_pay": {"asset": "USDC", "amount": "100.0"},
+        "you_receive": {"asset": "WETH", "amount": "0.04"}
+      },
+      "quote": {},
+      "requires_confirmation": true
+    },
+    {
+      "status": "confirmed",
+      "tx_hash": "0xabc123def456",
+      "explorer_url": "https://etherscan.io/tx/0xabc123def456",
+      "receipt": {"success": true},
+      "quote": {}
+    }
+  ]
+}

tests/fixtures/card_ui_schema/dev_tools__issue_resolver.json

@@ -0,0 +1,25 @@
+{
+  "samples": [
+    {
+      "status": "ready",
+      "action": "prepare",
+      "workflow_version": "1.0",
+      "issue": {
+        "url": "https://github.com/ARPAHLS/skillware/issues/199",
+        "api_url": "https://api.github.com/repos/ARPAHLS/skillware/issues/199",
+        "owner": "ARPAHLS",
+        "repo": "skillware",
+        "number": 199
+      },
+      "repository": {
+        "html_url": "https://github.com/ARPAHLS/skillware",
+        "api_url": "https://api.github.com/repos/ARPAHLS/skillware"
+      },
+      "auth": {
+        "token_provided": true,
+        "note": "Include the Authorization header."
+      },
+      "next_step": "Call action workflow_overview."
+    }
+  ]
+}

tests/fixtures/card_ui_schema/finance__uk_companies_house_handler.json

@@ -0,0 +1,14 @@
+{
+  "samples": [
+    {
+      "status": "ready",
+      "source": "companies_house_api",
+      "fetched_at": "2026-07-19T12:00:00Z",
+      "company_number": "01234567",
+      "company_name": "Example Ltd",
+      "company_status": "active",
+      "company_type": "ltd",
+      "next_actions": ["get_officers"]
+    }
+  ]
+}