{"kind":"Skill","metadata":{"namespace":"community","name":"arize-prompt-optimization","version":"0.1.0"},"spec":{"description":"Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.","files":{"SKILL.md":"---\nname: arize-prompt-optimization\ndescription: Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.\nmetadata:\n  author: arize\n  version: \"1.0\"\ncompatibility: Requires the ax CLI and a configured Arize profile.\n---\n\n# Arize Prompt Optimization Skill\n\n\u003e **`SPACE`** — All `--space` flags and the `ARIZE_SPACE` env var accept a space **name** (e.g., `my-workspace`) or a base64 space **ID** (e.g., `U3BhY2U6...`). Find yours with `ax spaces list`.\n\n## Concepts\n\n### Where Prompts Live in Trace Data\n\nLLM applications emit spans following OpenInference semantic conventions. Prompts are stored in different span attributes depending on the span kind and instrumentation:\n\n| Column | What it contains | When to use |\n|--------|-----------------|-------------|\n| `attributes.llm.input_messages` | Structured chat messages (system, user, assistant, tool) in role-based format | **Primary source** for chat-based LLM prompts |\n| `attributes.llm.input_messages.roles` | Array of roles: `system`, `user`, `assistant`, `tool` | Extract individual message roles |\n| `attributes.llm.input_messages.contents` | Array of message content strings | Extract message text |\n| `attributes.input.value` | Serialized prompt or user question (generic, all span kinds) | Fallback when structured messages are not available |\n| `attributes.llm.prompt_template.template` | Template with `{variable}` placeholders (e.g., `\"Answer {question} using {context}\"`) | When the app uses prompt templates |\n| `attributes.llm.prompt_template.variables` | Template variable values (JSON object) | See what values were substituted into the template |\n| `attributes.output.value` | Model response text | See what the LLM produced |\n| `attributes.llm.output_messages` | Structured model output (including tool calls) | Inspect tool-calling responses |\n\n### Finding Prompts by Span Kind\n\n- **LLM span** (`attributes.openinference.span.kind = 'LLM'`): Check `attributes.llm.input_messages` for structured chat messages, OR `attributes.input.value` for a serialized prompt. Check `attributes.llm.prompt_template.template` for the template.\n- **Chain/Agent span**: `attributes.input.value` contains the user's question. The actual LLM prompt lives on **child LLM spans** -- navigate down the trace tree.\n- **Tool span**: `attributes.input.value` has tool input, `attributes.output.value` has tool result. Not typically where prompts live.\n\n### Performance Signal Columns\n\nThese columns carry the feedback data used for optimization:\n\n| Column pattern | Source | What it tells you |\n|---------------|--------|-------------------|\n| `annotation.\u003cname\u003e.label` | Human reviewers | Categorical grade (e.g., `correct`, `incorrect`, `partial`) |\n| `annotation.\u003cname\u003e.score` | Human reviewers | Numeric quality score (e.g., 0.0 - 1.0) |\n| `annotation.\u003cname\u003e.text` | Human reviewers | Freeform explanation of the grade |\n| `eval.\u003cname\u003e.label` | LLM-as-judge evals | Automated categorical assessment |\n| `eval.\u003cname\u003e.score` | LLM-as-judge evals | Automated numeric score |\n| `eval.\u003cname\u003e.explanation` | LLM-as-judge evals | Why the eval gave that score -- **most valuable for optimization** |\n| `attributes.input.value` | Trace data | What went into the LLM |\n| `attributes.output.value` | Trace data | What the LLM produced |\n| `{experiment_name}.output` | Experiment runs | Output from a specific experiment |\n\n## Prerequisites\n\nProceed directly with the task — run the `ax` command you need. Do NOT check versions, env vars, or profiles upfront.\n\nIf an `ax` command fails, troubleshoot based on the error:\n- `command not found` or version error → see references/ax-setup.md\n- `401 Unauthorized` / missing API key → run `ax profiles show` to inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it. If the user doesn't have their key, direct them to https://app.arize.com/admin \u003e API Keys\n- Space unknown → run `ax spaces list` to pick by name, or ask the user\n- Project unclear → ask the user, or run `ax projects list -o json --limit 100` and present as selectable options\n- LLM provider call fails (missing OPENAI_API_KEY / ANTHROPIC_API_KEY) → run `ax ai-integrations list --space SPACE` to check for platform-managed credentials. If none exist, ask the user to provide the key or create an integration via the **arize-ai-provider-integration** skill\n- **Security:** Never read `.env` files or search the filesystem for credentials. Use `ax profiles` for Arize credentials and `ax ai-integrations` for LLM provider keys. If credentials are not available through these channels, ask the user.\n\n## Phase 1: Extract the Current Prompt\n\n### Find LLM spans containing prompts\n\n```bash\n# Sample LLM spans (where prompts live)\nax spans export PROJECT --filter \"attributes.openinference.span.kind = 'LLM'\" -l 10 --stdout\n\n# Filter by model\nax spans export PROJECT --filter \"attributes.llm.model_name = 'gpt-4o'\" -l 10 --stdout\n\n# Filter by span name (e.g., a specific LLM call)\nax spans export PROJECT --filter \"name = 'ChatCompletion'\" -l 10 --stdout\n```\n\n### Export a trace to inspect prompt structure\n\n```bash\n# Export all spans in a trace\nax spans export PROJECT --trace-id TRACE_ID\n\n# Export a single span\nax spans export PROJECT --span-id SPAN_ID\n```\n\n### Extract prompts from exported JSON\n\n```bash\n# Extract structured chat messages (system + user + assistant)\njq '.[0] | {\n  messages: .attributes.llm.input_messages,\n  model: .attributes.llm.model_name\n}' trace_*/spans.json\n\n# Extract the system prompt specifically\njq '[.[] | select(.attributes.llm.input_messages.roles[]? == \"system\")] | .[0].attributes.llm.input_messages' trace_*/spans.json\n\n# Extract prompt template and variables\njq '.[0].attributes.llm.prompt_template' trace_*/spans.json\n\n# Extract from input.value (fallback for non-structured prompts)\njq '.[0].attributes.input.value' trace_*/spans.json\n```\n\n### Reconstruct the prompt as messages\n\nOnce you have the span data, reconstruct the prompt as a messages array:\n\n```json\n[\n  {\"role\": \"system\", \"content\": \"You are a helpful assistant that...\"},\n  {\"role\": \"user\", \"content\": \"Given {input}, answer the question: {question}\"}\n]\n```\n\nIf the span has `attributes.llm.prompt_template.template`, the prompt uses variables. Preserve these placeholders (`{variable}` or `{{variable}}`) -- they are substituted at runtime.\n\n## Phase 2: Gather Performance Data\n\n### From traces (production feedback)\n\n```bash\n# Find error spans -- these indicate prompt failures\nax spans export PROJECT \\\n  --filter \"status_code = 'ERROR' AND attributes.openinference.span.kind = 'LLM'\" \\\n  -l 20 --stdout\n\n# Find spans with low eval scores\nax spans export PROJECT \\\n  --filter \"annotation.correctness.label = 'incorrect'\" \\\n  -l 20 --stdout\n\n# Find spans with high latency (may indicate overly complex prompts)\nax spans export PROJECT \\\n  --filter \"attributes.openinference.span.kind = 'LLM' AND latency_ms \u003e 10000\" \\\n  -l 20 --stdout\n\n# Export error traces for detailed inspection\nax spans export PROJECT --trace-id TRACE_ID\n```\n\n### From datasets and experiments\n\n```bash\n# Export a dataset (ground truth examples)\nax datasets export DATASET_NAME --space SPACE\n# -\u003e dataset_*/examples.json\n\n# Export experiment results (what the LLM produced)\nax experiments export EXPERIMENT_NAME --dataset DATASET_NAME --space SPACE\n# -\u003e experiment_*/runs.json\n```\n\n### Merge dataset + experiment for analysis\n\nJoin the two files by `example_id` to see inputs alongside outputs and evaluations:\n\n```bash\n# Count examples and runs\njq 'length' dataset_*/examples.json\njq 'length' experiment_*/runs.json\n\n# View a single joined record\njq -s '\n  .[0] as $dataset |\n  .[1][0] as $run |\n  ($dataset[] | select(.id == $run.example_id)) as $example |\n  {\n    input: $example,\n    output: $run.output,\n    evaluations: $run.evaluations\n  }\n' dataset_*/examples.json experiment_*/runs.json\n\n# Find failed examples (where eval score \u003c threshold)\njq '[.[] | select(.evaluations.correctness.score \u003c 0.5)]' experiment_*/runs.json\n```\n\n### Identify what to optimize\n\nLook for patterns across failures:\n\n1. **Compare outputs to ground truth**: Where does the LLM output differ from expected?\n2. **Read eval explanations**: `eval.*.explanation` tells you WHY something failed\n3. **Check annotation text**: Human feedback describes specific issues\n4. **Look for verbosity mismatches**: If outputs are too long/short vs ground truth\n5. **Check format compliance**: Are outputs in the expected format?\n\n## Phase 3: Optimize the Prompt\n\n### The Optimization Meta-Prompt\n\nUse this template to generate an improved version of the prompt. Fill in the three placeholders and send it to your LLM (GPT-4o, Claude, etc.):\n\n````\nYou are an expert in prompt optimization. Given the original baseline prompt\nand the associated performance data (inputs, outputs, evaluation labels, and\nexplanations), generate a revised version that improves results.\n\nORIGINAL BASELINE PROMPT\n========================\n\n{PASTE_ORIGINAL_PROMPT_HERE}\n\n========================\n\nPERFORMANCE DATA\n================\n\nThe following records show how the current prompt performed. Each record\nincludes the input, the LLM output, and evaluation feedback:\n\n{PASTE_RECORDS_HERE}\n\n================\n\nHOW TO USE THIS DATA\n\n1. Compare outputs: Look at what the LLM generated vs what was expected\n2. Review eval scores: Check which examples scored poorly and why\n3. Examine annotations: Human feedback shows what worked and what didn't\n4. Identify patterns: Look for common issues across multiple examples\n5. Focus on failures: The rows where the output DIFFERS from the expected\n   value are the ones that need fixing\n\nALIGNMENT STRATEGY\n\n- If outputs have extra text or reasoning not present in the ground truth,\n  remove instructions that encourage explanation or verbose reasoning\n- If outputs are missing information, add instructions to include it\n- If outputs are in the wrong format, add explicit format instructions\n- Focus on the rows where the output differs from the target -- these are\n  the failures to fix\n\nRULES\n\nMaintain Structure:\n- Use the same template variables as the current prompt ({var} or {{var}})\n- Don't change sections that are already working\n- Preserve the exact return format instructions from the original prompt\n\nAvoid Overfitting:\n- DO NOT copy examples verbatim into the prompt\n- DO NOT quote specific test data outputs exactly\n- INSTEAD: Extract the ESSENCE of what makes good vs bad outputs\n- INSTEAD: Add general guidelines and principles\n- INSTEAD: If adding few-shot examples, create SYNTHETIC examples that\n  demonstrate the principle, not real data from above\n\nGoal: Create a prompt that generalizes well to new inputs, not one that\nmemorizes the test data.\n\nOUTPUT FORMAT\n\nReturn the revised prompt as a JSON array of messages:\n\n[\n  {\"role\": \"system\", \"content\": \"...\"},\n  {\"role\": \"user\", \"content\": \"...\"}\n]\n\nAlso provide a brief reasoning section (bulleted list) explaining:\n- What problems you found\n- How the revised prompt addresses each one\n````\n\n### Preparing the performance data\n\nFormat the records as a JSON array before pasting into the template:\n\n```bash\n# From dataset + experiment: join and select relevant columns\njq -s '\n  .[0] as $ds |\n  [.[1][] | . as $run |\n    ($ds[] | select(.id == $run.example_id)) as $ex |\n    {\n      input: $ex.input,\n      expected: $ex.expected_output,\n      actual_output: $run.output,\n      eval_score: $run.evaluations.correctness.score,\n      eval_label: $run.evaluations.correctness.label,\n      eval_explanation: $run.evaluations.correctness.explanation\n    }\n  ]\n' dataset_*/examples.json experiment_*/runs.json\n\n# From exported spans: extract input/output pairs with annotations\njq '[.[] | select(.attributes.openinference.span.kind == \"LLM\") | {\n  input: .attributes.input.value,\n  output: .attributes.output.value,\n  status: .status_code,\n  model: .attributes.llm.model_name\n}]' trace_*/spans.json\n```\n\n### Applying the revised prompt\n\nAfter the LLM returns the revised messages array:\n\n1. Compare the original and revised prompts side by side\n2. Verify all template variables are preserved\n3. Check that format instructions are intact\n4. Test on a few examples before full deployment\n\n## Phase 4: Iterate\n\n### The optimization loop\n\n```\n1. Extract prompt    -\u003e Phase 1 (once)\n2. Run experiment    -\u003e ax experiments create ...\n3. Export results    -\u003e ax experiments export EXPERIMENT_NAME --dataset DATASET_NAME --space SPACE\n4. Analyze failures  -\u003e jq to find low scores\n5. Run meta-prompt   -\u003e Phase 3 with new failure data\n6. Apply revised prompt\n7. Repeat from step 2\n```\n\n### Measure improvement\n\n```bash\n# Compare scores across experiments\n# Experiment A (baseline)\njq '[.[] | .evaluations.correctness.score] | add / length' experiment_a/runs.json\n\n# Experiment B (optimized)\njq '[.[] | .evaluations.correctness.score] | add / length' experiment_b/runs.json\n\n# Find examples that flipped from fail to pass\njq -s '\n  [.[0][] | select(.evaluations.correctness.label == \"incorrect\")] as $fails |\n  [.[1][] | select(.evaluations.correctness.label == \"correct\") |\n    select(.example_id as $id | $fails | any(.example_id == $id))\n  ] | length\n' experiment_a/runs.json experiment_b/runs.json\n```\n\n### A/B compare two prompts\n\n1. Create two experiments against the same dataset, each using a different prompt version\n2. Export both: `ax experiments export EXP_A` and `ax experiments export EXP_B`\n3. Compare average scores, failure rates, and specific example flips\n4. Check for regressions -- examples that passed with prompt A but fail with prompt B\n\n## Prompt Engineering Best Practices\n\nApply these when writing or revising prompts:\n\n| Technique | When to apply | Example |\n|-----------|--------------|---------|\n| Clear, detailed instructions | Output is vague or off-topic | \"Classify the sentiment as exactly one of: positive, negative, neutral\" |\n| Instructions at the beginning | Model ignores later instructions | Put the task description before examples |\n| Step-by-step breakdowns | Complex multi-step processes | \"First extract entities, then classify each, then summarize\" |\n| Specific personas | Need consistent style/tone | \"You are a senior financial analyst writing for institutional investors\" |\n| Delimiter tokens | Sections blend together | Use `---`, `###`, or XML tags to separate input from instructions |\n| Few-shot examples | Output format needs clarification | Show 2-3 synthetic input/output pairs |\n| Output length specifications | Responses are too long or short | \"Respond in exactly 2-3 sentences\" |\n| Reasoning instructions | Accuracy is critical | \"Think step by step before answering\" |\n| \"I don't know\" guidelines | Hallucination is a risk | \"If the answer is not in the provided context, say 'I don't have enough information'\" |\n\n### Variable preservation\n\nWhen optimizing prompts that use template variables:\n\n- **Single braces** (`{variable}`): Python f-string / Jinja style. Most common in Arize.\n- **Double braces** (`{{variable}}`): Mustache style. Used when the framework requires it.\n- Never add or remove variable placeholders during optimization\n- Never rename variables -- the runtime substitution depends on exact names\n- If adding few-shot examples, use literal values, not variable placeholders\n\n## Workflows\n\n### Optimize a prompt from a failing trace\n\n1. Find failing traces:\n   ```bash\n   ax traces list PROJECT --filter \"status_code = 'ERROR'\" --limit 5\n   ```\n2. Export the trace:\n   ```bash\n   ax spans export PROJECT --trace-id TRACE_ID\n   ```\n3. Extract the prompt from the LLM span:\n   ```bash\n   jq '[.[] | select(.attributes.openinference.span.kind == \"LLM\")][0] | {\n     messages: .attributes.llm.input_messages,\n     template: .attributes.llm.prompt_template,\n     output: .attributes.output.value,\n     error: .attributes.exception.message\n   }' trace_*/spans.json\n   ```\n4. Identify what failed from the error message or output\n5. Fill in the optimization meta-prompt (Phase 3) with the prompt and error context\n6. Apply the revised prompt\n\n### Optimize using a dataset and experiment\n\n1. Find the dataset and experiment:\n   ```bash\n   ax datasets list --space SPACE\n   ax experiments list --dataset DATASET_NAME --space SPACE\n   ```\n2. Export both:\n   ```bash\n   ax datasets export DATASET_NAME --space SPACE\n   ax experiments export EXPERIMENT_NAME --dataset DATASET_NAME --space SPACE\n   ```\n3. Prepare the joined data for the meta-prompt\n4. Run the optimization meta-prompt\n5. Create a new experiment with the revised prompt to measure improvement\n\n### Debug a prompt that produces wrong format\n\n1. Export spans where the output format is wrong:\n   ```bash\n   ax spans export PROJECT \\\n     --filter \"attributes.openinference.span.kind = 'LLM' AND annotation.format.label = 'incorrect'\" \\\n     -l 10 --stdout \u003e bad_format.json\n   ```\n2. Look at what the LLM is producing vs what was expected\n3. Add explicit format instructions to the prompt (JSON schema, examples, delimiters)\n4. Common fix: add a few-shot example showing the exact desired output format\n\n### Reduce hallucination in a RAG prompt\n\n1. Find traces where the model hallucinated:\n   ```bash\n   ax spans export PROJECT \\\n     --filter \"annotation.faithfulness.label = 'unfaithful'\" \\\n     -l 20 --stdout\n   ```\n2. Export and inspect the retriever + LLM spans together:\n   ```bash\n   ax spans export PROJECT --trace-id TRACE_ID\n   jq '[.[] | {kind: .attributes.openinference.span.kind, name, input: .attributes.input.value, output: .attributes.output.value}]' trace_*/spans.json\n   ```\n3. Check if the retrieved context actually contained the answer\n4. Add grounding instructions to the system prompt: \"Only use information from the provided context. If the answer is not in the context, say so.\"\n\n## Troubleshooting\n\n| Problem | Solution |\n|---------|----------|\n| `ax: command not found` | See references/ax-setup.md |\n| `No profile found` | No profile is configured. See references/ax-profiles.md to create one. |\n| No `input_messages` on span | Check span kind -- Chain/Agent spans store prompts on child LLM spans, not on themselves |\n| Prompt template is `null` | Not all instrumentations emit `prompt_template`. Use `input_messages` or `input.value` instead |\n| Variables lost after optimization | Verify the revised prompt preserves all `{var}` placeholders from the original |\n| Optimization makes things worse | Check for overfitting -- the meta-prompt may have memorized test data. Ensure few-shot examples are synthetic |\n| No eval/annotation columns | Run evaluations first (via Arize UI or SDK), then re-export |\n| Experiment output column not found | The column name is `{experiment_name}.output` -- check exact experiment name via `ax experiments get` |\n| `jq` errors on span JSON | Ensure you're targeting the correct file path (e.g., `trace_*/spans.json`) |\n","references/ax-profiles.md":"# ax Profile Setup\n\nConsult this when authentication fails (401, missing profile, missing API key). Do NOT run these checks proactively.\n\nUse this when there is no profile, or a profile has incorrect settings (wrong API key, wrong region, etc.).\n\n## 1. Inspect the current state\n\n```bash\nax profiles show\n```\n\nLook at the output to understand what's configured:\n- `API Key: (not set)` or missing → key needs to be created/updated\n- No profile output or \"No profiles found\" → no profile exists yet\n- Connected but getting `401 Unauthorized` → key is wrong or expired\n- Connected but wrong endpoint/region → region needs to be updated\n\n## 2. Fix a misconfigured profile\n\nIf a profile exists but one or more settings are wrong, patch only what's broken.\n\n**Never pass a raw API key value as a flag.** Always reference it via the `ARIZE_API_KEY` environment variable. If the variable is not already set in the shell, instruct the user to set it first, then run the command:\n\n```bash\n# If ARIZE_API_KEY is already exported in the shell:\nax profiles update --api-key $ARIZE_API_KEY\n\n# Fix the region (no secret involved — safe to run directly)\nax profiles update --region us-east-1b\n\n# Fix both at once\nax profiles update --api-key $ARIZE_API_KEY --region us-east-1b\n```\n\n`update` only changes the fields you specify — all other settings are preserved. If no profile name is given, the active profile is updated.\n\n## 3. Create a new profile\n\nIf no profile exists, or if the existing profile needs to point to a completely different setup (different org, different region):\n\n**Always reference the key via `$ARIZE_API_KEY`, never inline a raw value.**\n\n```bash\n# Requires ARIZE_API_KEY to be exported in the shell first\nax profiles create --api-key $ARIZE_API_KEY\n\n# Create with a region\nax profiles create --api-key $ARIZE_API_KEY --region us-east-1b\n\n# Create a named profile\nax profiles create work --api-key $ARIZE_API_KEY --region us-east-1b\n```\n\nTo use a named profile with any `ax` command, add `-p NAME`:\n```bash\nax spans export PROJECT -p work\n```\n\n## 4. Getting the API key\n\n**Never ask the user to paste their API key into the chat. Never log, echo, or display an API key value.**\n\nIf `ARIZE_API_KEY` is not already set, instruct the user to export it in their shell:\n\n```bash\nexport ARIZE_API_KEY=\"...\"   # user pastes their key here in their own terminal\n```\n\nThey can find their key at https://app.arize.com/admin \u003e API Keys. Recommend they create a **scoped service key** (not a personal user key) — service keys are not tied to an individual account and are safer for programmatic use. Keys are space-scoped — make sure they copy the key for the correct space.\n\nOnce the user confirms the variable is set, proceed with `ax profiles create --api-key $ARIZE_API_KEY` or `ax profiles update --api-key $ARIZE_API_KEY` as described above.\n\n## 5. Verify\n\nAfter any create or update:\n\n```bash\nax profiles show\n```\n\nConfirm the API key and region are correct, then retry the original command.\n\n## Space\n\nThere is no profile flag for space. Save it as an environment variable — accepts a space **name** (e.g., `my-workspace`) or a base64 space **ID** (e.g., `U3BhY2U6...`). Find yours with `ax spaces list -o json`.\n\n**macOS/Linux** — add to `~/.zshrc` or `~/.bashrc`:\n```bash\nexport ARIZE_SPACE=\"my-workspace\"    # name or base64 ID\n```\nThen `source ~/.zshrc` (or restart terminal).\n\n**Windows (PowerShell):**\n```powershell\n[System.Environment]::SetEnvironmentVariable('ARIZE_SPACE', 'my-workspace', 'User')\n```\nRestart terminal for it to take effect.\n\n## Save Credentials for Future Use\n\nAt the **end of the session**, if the user manually provided any credentials during this conversation **and** those values were NOT already loaded from a saved profile or environment variable, offer to save them.\n\n**Skip this entirely if:**\n- The API key was already loaded from an existing profile or `ARIZE_API_KEY` env var\n- The space was already set via `ARIZE_SPACE` env var\n- The user only used base64 project IDs (no space was needed)\n\n**How to offer:** Use **AskQuestion**: *\"Would you like to save your Arize credentials so you don't have to enter them next time?\"* with options `\"Yes, save them\"` / `\"No thanks\"`.\n\n**If the user says yes:**\n\n1. **API key** — Run `ax profiles show` to check the current state. Then run `ax profiles create --api-key $ARIZE_API_KEY` or `ax profiles update --api-key $ARIZE_API_KEY` (the key must already be exported as an env var — never pass a raw key value).\n\n2. **Space** — See the Space section above to persist it as an environment variable.\n","references/ax-setup.md":"# ax CLI — Troubleshooting\n\nConsult this only when an `ax` command fails. Do NOT run these checks proactively.\n\n## Check version first\n\nIf `ax` is installed (not `command not found`), always run `ax --version` before investigating further. The version must be `0.14.0` or higher — many errors are caused by an outdated install. If the version is too old, see **Version too old** below.\n\n## `ax: command not found`\n\n**macOS/Linux:**\n1. Check common locations: `~/.local/bin/ax`, `~/Library/Python/*/bin/ax`\n2. Install: `uv tool install arize-ax-cli` (preferred), `pipx install arize-ax-cli`, or `pip install arize-ax-cli`\n3. Add to PATH if needed: `export PATH=\"$HOME/.local/bin:$PATH\"`\n\n**Windows (PowerShell):**\n1. Check: `Get-Command ax` or `where.exe ax`\n2. Common locations: `%APPDATA%\\Python\\Scripts\\ax.exe`, `%LOCALAPPDATA%\\Programs\\Python\\Python*\\Scripts\\ax.exe`\n3. Install: `pip install arize-ax-cli`\n4. Add to PATH: `$env:PATH = \"$env:APPDATA\\Python\\Scripts;$env:PATH\"`\n\n## Version too old (below 0.14.0)\n\nUpgrade: `uv tool install --force --reinstall arize-ax-cli`, `pipx upgrade arize-ax-cli`, or `pip install --upgrade arize-ax-cli`\n\n## SSL/certificate error\n\n- macOS: `export SSL_CERT_FILE=/etc/ssl/cert.pem`\n- Linux: `export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt`\n- Fallback: `export SSL_CERT_FILE=$(python -c \"import certifi; print(certifi.where())\")`\n\n## Subcommand not recognized\n\nUpgrade ax (see above) or use the closest available alternative.\n\n## Still failing\n\nStop and ask the user for help.\n"},"import":{"commit_sha":"541b7819d8c3545c6df122491af4fa1eae415779","imported_at":"2026-05-18T20:05:35Z","license_text":"MIT License\n\nCopyright GitHub, Inc.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.","owner":"github","repo":"github/awesome-copilot","source_url":"https://github.com/github/awesome-copilot/tree/541b7819d8c3545c6df122491af4fa1eae415779/plugins/arize-ax/skills/arize-prompt-optimization"}},"content_hash":[222,188,215,60,173,150,10,250,53,136,155,232,38,245,97,195,52,133,219,252,38,164,172,247,248,69,229,186,254,231,55,31],"trust_level":"unsigned","yanked":false}