{"kind":"Skill","metadata":{"namespace":"community","name":"openclaw-pre-release-plugin-testing","version":"0.1.0"},"spec":{"description":"Plan and run pre-release OpenClaw plugin validation across bundled plugins, package artifacts, lifecycle commands, doctor/fix, config round-trip, gateway startup, SDK compatibility, Docker E2E, Package Acceptance, and Testbox proof.","files":{"SKILL.md":"---\nname: openclaw-pre-release-plugin-testing\ndescription: Plan and run pre-release OpenClaw plugin validation across bundled plugins, package artifacts, lifecycle commands, doctor/fix, config round-trip, gateway startup, SDK compatibility, Docker E2E, Package Acceptance, and Testbox proof.\n---\n\n# OpenClaw Pre-Release Plugin Testing\n\nUse this skill when the user asks for plugin release confidence, plugin lifecycle\nsweeps, package-artifact plugin proof, or \"what else should we test before\nrelease?\" It complements `openclaw-testing`; use that skill too when choosing\nthe cheapest safe runner or debugging a failing lane.\n\n## Goal\n\nProve the plugin system as a product surface, not just as source tests:\n\n- bundled plugin lifecycle: install, inspect, enable, disable, uninstall\n- package artifact behavior from a clean `HOME`\n- doctor/fix/config validation and idempotence\n- config discovery and config round-trip\n- status/log visibility and diagnostics\n- gateway startup/bootstrap with plugin metadata snapshots\n- public SDK compatibility for real external plugins\n- live-ish provider/channel probes only when safe credentials exist\n\n## First Checks\n\nFrom the OpenClaw repo root:\n\n```bash\npnpm docs:list\ngit status --short --branch\nreadlink node_modules\npnpm changed:lanes --json\n```\n\nIn Codex worktrees under `.codex/worktrees`, `node_modules` must be a symlink to\nthe main OpenClaw checkout. Do not run `pnpm install` there. For broad or\npackage-heavy proof, use Blacksmith Testbox or GitHub Actions.\n\n## Runner Choice\n\nPrefer this order:\n\n1. **GitHub Package Acceptance** for installable-package product proof.\n2. **`ci-build-artifacts-testbox.yml` Testbox** when Docker/package lanes need\n   seeded `dist`, `dist-runtime`, and package caches.\n3. **`ci-check-testbox.yml` Testbox** for source checks, targeted Vitest,\n   package-boundary checks, or focused Docker lanes.\n4. **Local targeted commands only** for small format/static/unit probes.\n\nAvoid long package Docker runs from a stale sparse worktree. If Testbox sync\nreports hundreds of changed files or starts deleting package inputs, stop and\nwarm a fresh box from current `main`, or switch to Package Acceptance.\n\n## Existing Baseline\n\nRun or verify these before inventing new coverage:\n\n```bash\nOPENCLAW_TESTBOX=1 pnpm check:changed\npnpm run test:extensions:package-boundary:canary\npnpm run test:extensions:package-boundary:compile\npnpm test:docker:plugins\nOPENCLAW_PLUGINS_E2E_CLAWHUB=0 pnpm test:docker:plugins\npnpm test:docker:plugin-update\npnpm test:docker:bundled-channel-deps:fast\n```\n\nFor full bundled install/uninstall proof, shard the packaged sweep:\n\n```bash\nOPENCLAW_BUNDLED_PLUGIN_SWEEP_TOTAL=8 \\\nOPENCLAW_BUNDLED_PLUGIN_SWEEP_INDEX=\u003c0-7\u003e \\\npnpm test:docker:bundled-plugin-install-uninstall\n```\n\nExpected current packaged scope: 116 public bundled plugins over shards `0-7`.\nPrivate QA plugins are source-mode only unless a package explicitly includes\nthem.\n\n## Confidence Matrix\n\nUse this matrix for pre-release signoff. Record pass/fail, run URL/Testbox ID,\npackage SHA/version, and skipped-live reason.\n\n| Surface | Proof | Preferred runner |\n| --- | --- | --- |\n| Package artifact | Package Acceptance `suite_profile=package` or custom lanes | GitHub Actions |\n| Bundled lifecycle | 8-shard `test:docker:bundled-plugin-install-uninstall` | Testbox or release Docker |\n| External plugins | `test:docker:plugins` and `plugins-offline` | Testbox/package acceptance |\n| Update no-op | `test:docker:plugin-update` | Testbox/package acceptance |\n| Channel runtime deps | `test:docker:bundled-channel-deps:fast` plus key channels | Testbox/package acceptance |\n| Doctor/fix | seeded bad configs + `doctor --fix --non-interactive` | new Docker/Testbox harness |\n| Config round-trip | `config set/get`, inspect, doctor, reload, diff hash | new Docker/Testbox harness |\n| Gateway bootstrap | clean `HOME`, plugin groups enabled/disabled, status JSON | new Docker/Testbox harness |\n| SDK compatibility | directory, tgz, and `file:` external plugins using SDK subpaths | `test:docker:plugins` plus new smoke |\n| Live-ish | redacted provider/channel probes only for present env | Testbox live lanes |\n\n## Package Acceptance Plan\n\nUse this when validating a release branch, beta, or candidate package:\n\n```bash\ngh workflow run package-acceptance.yml \\\n  --repo openclaw/openclaw \\\n  --ref main \\\n  -f workflow_ref=main \\\n  -f source=ref \\\n  -f package_ref=\u003cbranch-or-sha\u003e \\\n  -f suite_profile=custom \\\n  -f docker_lanes='plugins-offline plugin-update bundled-channel-deps-compat doctor-switch update-channel-switch config-reload mcp-channels npm-onboard-channel-agent' \\\n  -f telegram_mode=mock-openai\n```\n\nUse `source=npm -f package_spec=openclaw@beta` for published beta proof. Keep\n`workflow_ref` as trusted current harness code unless the release process says\notherwise.\n\n## New Testbox Harness Plan\n\nIf more certainty is needed, add or run a `plugin-lifecycle-matrix` Docker lane\nthat uses one package tarball and sharded plugin lists. Per plugin:\n\n1. Start with a clean `HOME`.\n2. Capture `plugins list --json`.\n3. `plugins install \u003cid\u003e`.\n4. `plugins inspect \u003cid\u003e --json`.\n5. `plugins disable \u003cid\u003e`, then assert disabled visibility.\n6. `plugins enable \u003cid\u003e`, except config-required plugins without config.\n7. `plugins registry --refresh`.\n8. `doctor --non-interactive`.\n9. `plugins uninstall \u003cid\u003e --force`.\n10. Assert no config entry, allow/deny residue, install record, managed dir, or\n    bundled `dist/extensions/...` load path remains.\n11. Assert diagnostics contain no `level: \"error\"` and output redacts\n    secret-looking values.\n\nKeep `memory-lancedb` special: it is config-required. First assert install does\nnot enable it without embedding config, then run a second configured case.\n\n## Doctor/Fix Matrix\n\nSeed bad states and require `doctor --fix --non-interactive` to repair them,\nthen run doctor again and require idempotence:\n\n- stale `plugins.allow`\n- stale `plugins.entries`\n- stale channel config for missing channel plugin\n- invalid `plugins.entries.\u003cid\u003e.config`\n- packaged bundled path in `plugins.load.paths`\n- legacy `plugins.installs`\n- disabled channel/plugin config that must not stage runtime deps\n- root-owned global package tree that must remain unmodified\n\n## Gateway Bootstrap Matrix\n\nStart packaged OpenClaw in Docker with clean state:\n\n- provider plugins enabled, no credentials: ready with warnings, no crash\n- channel plugins configured disabled: no runtime deps staged\n- startup-activation plugins enabled: ready and reflected in status\n- invalid single plugin config: bad plugin skipped/quarantined, others remain\n\nAssert:\n\n- gateway reaches ready\n- `openclaw status --json` includes plugin diagnostics\n- `openclaw plugins inspect --all --json` is parseable\n- package tree is not mutated\n- logs contain no raw tokens\n\n## Config Round-Trip Representatives\n\nUse representative plugin families instead of every plugin for deep config\nround-trip:\n\n- providers: `openai`, `anthropic`, `mistral`, `openrouter`\n- channels: `telegram`, `discord`, `slack`, `whatsapp`\n- memory: `memory-lancedb`\n- feature/runtime: `browser`, `acpx`, `tokenjuice`\n\nFor each representative:\n\n1. Write config through CLI when possible.\n2. Read it back through `config get` or JSON.\n3. Run `plugins inspect`.\n4. Run `doctor --non-interactive`.\n5. Trigger gateway config reload if applicable.\n6. Compare config hash before/after no-op commands.\n\n## External SDK Smoke\n\nIn a package Docker lane, create tiny external plugins and install them from:\n\n- local directory\n- `.tgz`\n- `file:` npm spec\n\nCover CJS and ESM shapes, plus at least one plugin importing focused\n`openclaw/plugin-sdk/*` subpaths. Assert `plugins inspect` sees its tool,\ngateway method, CLI command, or service.\n\n## Live-Ish Probe Rules\n\nBefore live-ish work, source allowed env in Testbox and generate a redacted\navailability matrix: present/missing only, never values.\n\nOnly run probes for credentials that exist. Prefer auth/catalog/status probes\nover sending user-visible messages. If a probe might contact an external user,\nchannel, or workspace, stop and ask the user.\n\n## Reporting\n\nReport in this shape:\n\n```text\npackage/ref:\ntbx ids / run urls:\nmatrix:\n  bundled lifecycle:\n  package acceptance:\n  doctor/fix:\n  gateway bootstrap:\n  config round-trip:\n  sdk external:\n  live-ish:\nfailures:\nskips:\nnext highest-value gap:\n```\n\nSay clearly when a failure is Testbox sync/env damage rather than product\nbehavior, and prove that with a clean rerun or current-main comparison.\n","agents/openai.yaml":"interface:\n  display_name: \"OpenClaw Plugin Pre-Release Testing\"\n  short_description: \"Plan plugin release validation\"\n  default_prompt: \"Use $openclaw-pre-release-plugin-testing to plan or run pre-release OpenClaw plugin validation across package, lifecycle, doctor, gateway, SDK, and live-ish proof.\"\n"},"import":{"commit_sha":"424c6d0a5f4665b803ad6768d08b0be7659deaf4","imported_at":"2026-05-18T20:13:36Z","license_text":"MIT License\n\nCopyright (c) 2025 Peter Steinberger\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","owner":"openclaw","repo":"openclaw/openclaw","source_url":"https://github.com/openclaw/openclaw/tree/424c6d0a5f4665b803ad6768d08b0be7659deaf4/.agents/skills/openclaw-pre-release-plugin-testing"}},"content_hash":[25,181,240,178,205,52,85,241,145,145,187,66,157,96,113,127,219,203,110,243,211,166,51,41,31,194,12,174,15,225,145,16],"trust_level":"unsigned","yanked":false}