{"kind":"AgentDefinition","metadata":{"namespace":"community","name":"dataverse-python-best-practices","version":"0.1.0"},"spec":{"agents_md":"# Dataverse SDK for Python - Best Practices Guide\n\n## Overview\nProduction-ready patterns and best practices extracted from Microsoft's official PowerPlatform-DataverseClient-Python repository, examples, and recommended workflows.\n\n## 1. Installation \u0026 Environment Setup\n\n### Production Installation\n```bash\n# Install the published SDK from PyPI\npip install PowerPlatform-Dataverse-Client\n\n# Install Azure Identity for authentication\npip install azure-identity\n\n# Optional: pandas integration for data manipulation\npip install pandas\n```\n\n### Development Installation\n```bash\n# Clone the repository\ngit clone https://github.com/microsoft/PowerPlatform-DataverseClient-Python.git\ncd PowerPlatform-DataverseClient-Python\n\n# Install in editable mode for live development\npip install -e .\n\n# Install development dependencies\npip install pytest pytest-cov black isort mypy ruff\n```\n\n### Python Version Support\n- **Minimum**: Python 3.10\n- **Recommended**: Python 3.11+ for best performance\n- **Supported**: Python 3.10, 3.11, 3.12, 3.13, 3.14\n\n### Verify Installation\n```python\nfrom PowerPlatform.Dataverse import __version__\nfrom PowerPlatform.Dataverse.client import DataverseClient\nfrom azure.identity import InteractiveBrowserCredential\n\nprint(f\"SDK Version: {__version__}\")\nprint(\"Installation successful!\")\n```\n\n---\n\n## 2. Authentication Patterns\n\n### Interactive Development (Browser-Based)\n```python\nfrom azure.identity import InteractiveBrowserCredential\nfrom PowerPlatform.Dataverse.client import DataverseClient\n\ncredential = InteractiveBrowserCredential()\nclient = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential)\n```\n\n**When to use:** Local development, interactive testing, single-user scenarios.\n\n### Production (Client Secret)\n```python\nfrom azure.identity import ClientSecretCredential\nfrom PowerPlatform.Dataverse.client import DataverseClient\n\ncredential = ClientSecretCredential(\n    tenant_id=\"your-tenant-id\",\n    client_id=\"your-client-id\",\n    client_secret=\"your-client-secret\"\n)\nclient = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential)\n```\n\n**When to use:** Server-side applications, Azure automation, scheduled jobs.\n\n### Certificate-Based Authentication\n```python\nfrom azure.identity import ClientCertificateCredential\nfrom PowerPlatform.Dataverse.client import DataverseClient\n\ncredential = ClientCertificateCredential(\n    tenant_id=\"your-tenant-id\",\n    client_id=\"your-client-id\",\n    certificate_path=\"path/to/certificate.pem\"\n)\nclient = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential)\n```\n\n**When to use:** Highly secure environments, certificate-pinning requirements.\n\n### Azure CLI Authentication\n```python\nfrom azure.identity import AzureCliCredential\nfrom PowerPlatform.Dataverse.client import DataverseClient\n\ncredential = AzureCliCredential()\nclient = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential)\n```\n\n**When to use:** Local testing with Azure CLI installed, Azure DevOps pipelines.\n\n---\n\n## 3. Singleton Client Pattern\n\n**Best Practice**: Create one `DataverseClient` instance and reuse it throughout your application.\n\n```python\n# ❌ ANTI-PATTERN: Creating new clients repeatedly\ndef fetch_account(account_id):\n    credential = InteractiveBrowserCredential()\n    client = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential)\n    return client.get(\"account\", account_id)\n\n# ✅ PATTERN: Singleton client\nclass DataverseService:\n    _instance = None\n    \n    def __new__(cls):\n        if cls._instance is None:\n            credential = InteractiveBrowserCredential()\n            cls._instance = DataverseClient(\n                \"https://yourorg.crm.dynamics.com\", \n                credential\n            )\n        return cls._instance\n\n# Usage\nservice = DataverseService()\naccount = service.get(\"account\", account_id)\n```\n\n---\n\n## 4. Configuration Optimization\n\n### Connection Settings\n```python\nfrom PowerPlatform.Dataverse.core.config import DataverseConfig\nfrom PowerPlatform.Dataverse.client import DataverseClient\nfrom azure.identity import ClientSecretCredential\n\nconfig = DataverseConfig(\n    language_code=1033,  # English (US)\n    # Note: http_retries, http_backoff, http_timeout are reserved for internal use\n)\n\ncredential = ClientSecretCredential(tenant_id, client_id, client_secret)\nclient = DataverseClient(\"https://yourorg.crm.dynamics.com\", credential, config)\n```\n\n**Key configuration options:**\n- `language_code`: Language for API responses (default: 1033 for English)\n\n---\n\n## 5. CRUD Operations Best Practices\n\n### Create Operations\n\n#### Single Record\n```python\nrecord_data = {\n    \"name\": \"Contoso Ltd\",\n    \"telephone1\": \"555-0100\",\n    \"creditlimit\": 100000.00,\n}\ncreated_ids = client.create(\"account\", record_data)\nrecord_id = created_ids[0]\nprint(f\"Created: {record_id}\")\n```\n\n#### Bulk Create (Automatically Optimized)\n```python\n# SDK automatically uses CreateMultiple for arrays \u003e 1 record\nrecords = [\n    {\"name\": f\"Company {i}\", \"creditlimit\": 50000 + (i * 1000)}\n    for i in range(100)\n]\ncreated_ids = client.create(\"account\", records)\nprint(f\"Created {len(created_ids)} records\")\n```\n\n**Performance**: Bulk create is optimized internally; no manual batching required.\n\n### Read Operations\n\n#### Single Record by ID\n```python\naccount = client.get(\"account\", \"account-guid-here\")\nprint(account.get(\"name\"))\n```\n\n#### Query with Filtering \u0026 Selection\n```python\n# Returns paginated results (generator)\nfor page in client.get(\n    \"account\",\n    filter=\"creditlimit gt 50000\",\n    select=[\"name\", \"creditlimit\", \"telephone1\"],\n    orderby=\"name\",\n    top=100\n):\n    for account in page:\n        print(f\"{account['name']}: ${account['creditlimit']}\")\n```\n\n**Key parameters:**\n- `filter`: OData filter (must use **lowercase** logical names)\n- `select`: Fields to retrieve (improves performance)\n- `orderby`: Sort results\n- `top`: Max records per page (default: 5000)\n- `page_size`: Override page size for pagination\n\n#### SQL Queries (Read-Only)\n```python\n# SQL queries are read-only; use for complex analytics\nresults = client.query_sql(\"\"\"\n    SELECT TOP 10 name, creditlimit \n    FROM account \n    WHERE creditlimit \u003e 50000\n    ORDER BY name\n\"\"\")\n\nfor row in results:\n    print(f\"{row['name']}: ${row['creditlimit']}\")\n```\n\n**Limitations:**\n- Read-only (SELECT only, no DML)\n- Useful for complex joins and analytics\n- May be disabled by org policy\n\n### Update Operations\n\n#### Single Record\n```python\nclient.update(\"account\", \"account-guid\", {\n    \"creditlimit\": 150000.00,\n    \"name\": \"Updated Company Name\"\n})\n```\n\n#### Bulk Update (Broadcast Same Change)\n```python\n# Update all selected records with same data\naccount_ids = [\"id1\", \"id2\", \"id3\"]\nclient.update(\"account\", account_ids, {\n    \"industrycode\": 1,  # Retail\n    \"accountmanagerid\": \"manager-guid\"\n})\n```\n\n#### Paired Updates (1:1 Record Updates)\n```python\n# For different updates per record, send multiple calls\nupdates = {\n    \"id1\": {\"creditlimit\": 100000},\n    \"id2\": {\"creditlimit\": 200000},\n    \"id3\": {\"creditlimit\": 300000},\n}\nfor record_id, data in updates.items():\n    client.update(\"account\", record_id, data)\n```\n\n### Delete Operations\n\n#### Single Record\n```python\nclient.delete(\"account\", \"account-guid\")\n```\n\n#### Bulk Delete (Optimized)\n```python\n# SDK automatically uses BulkDelete for large lists\nrecord_ids = [\"id1\", \"id2\", \"id3\", ...]\nclient.delete(\"account\", record_ids, use_bulk_delete=True)\n```\n\n---\n\n## 6. Error Handling \u0026 Recovery\n\n### Exception Hierarchy\n```python\nfrom PowerPlatform.Dataverse.core.errors import (\n    DataverseError,           # Base class\n    ValidationError,          # Validation failures\n    MetadataError,           # Table/column operations\n    HttpError,               # HTTP-level errors\n    SQLParseError            # SQL query syntax errors\n)\n\ntry:\n    client.create(\"account\", {\"name\": None})  # Invalid\nexcept ValidationError as e:\n    print(f\"Validation failed: {e}\")\n    # Handle validation-specific logic\nexcept DataverseError as e:\n    print(f\"General SDK error: {e}\")\n    # Handle other SDK errors\n```\n\n### Retry Logic Pattern\n```python\nimport time\nfrom PowerPlatform.Dataverse.core.errors import HttpError\n\ndef create_with_retry(table_name, record_data, max_retries=3):\n    \"\"\"Create record with exponential backoff retry logic.\"\"\"\n    for attempt in range(max_retries):\n        try:\n            return client.create(table_name, record_data)\n        except HttpError as e:\n            if attempt == max_retries - 1:\n                raise\n            \n            # Exponential backoff: 1s, 2s, 4s\n            backoff_seconds = 2 ** attempt\n            print(f\"Attempt {attempt + 1} failed. Retrying in {backoff_seconds}s...\")\n            time.sleep(backoff_seconds)\n\n# Usage\ncreated_ids = create_with_retry(\"account\", {\"name\": \"Contoso\"})\n```\n\n### 429 (Request Rate Limit) Handling\n```python\nimport time\nfrom PowerPlatform.Dataverse.core.errors import HttpError\n\ntry:\n    accounts = client.get(\"account\", top=5000)\nexcept HttpError as e:\n    if \"429\" in str(e):\n        # Rate limited; wait and retry\n        print(\"Rate limited. Waiting 60 seconds...\")\n        time.sleep(60)\n        accounts = client.get(\"account\", top=5000)\n    else:\n        raise\n```\n\n---\n\n## 7. Table \u0026 Column Management\n\n### Create Custom Table\n```python\nfrom enum import IntEnum\n\nclass Priority(IntEnum):\n    LOW = 1\n    MEDIUM = 2\n    HIGH = 3\n\n# Define columns with types\ncolumns = {\n    \"new_Title\": \"string\",\n    \"new_Quantity\": \"int\",\n    \"new_Amount\": \"decimal\",\n    \"new_Completed\": \"bool\",\n    \"new_Priority\": Priority,  # Creates option set/picklist\n    \"new_CreatedDate\": \"datetime\"\n}\n\ntable_info = client.create_table(\n    \"new_CustomTable\",\n    primary_column_schema_name=\"new_Name\",\n    columns=columns\n)\n\nprint(f\"Created table: {table_info['table_schema_name']}\")\n```\n\n### Get Table Metadata\n```python\ntable_info = client.get_table_info(\"account\")\nprint(f\"Schema Name: {table_info['table_schema_name']}\")\nprint(f\"Logical Name: {table_info['table_logical_name']}\")\nprint(f\"Entity Set: {table_info['entity_set_name']}\")\nprint(f\"Primary ID: {table_info['primary_id_attribute']}\")\n```\n\n### List All Tables\n```python\ntables = client.list_tables()\nfor table in tables:\n    print(f\"{table['table_schema_name']} ({table['table_logical_name']})\")\n```\n\n### Column Management\n```python\n# Add columns to existing table\nclient.create_columns(\"new_CustomTable\", {\n    \"new_Status\": \"string\",\n    \"new_Priority\": \"int\"\n})\n\n# Delete columns\nclient.delete_columns(\"new_CustomTable\", [\"new_Status\", \"new_Priority\"])\n\n# Delete table\nclient.delete_table(\"new_CustomTable\")\n```\n\n---\n\n## 8. Paging \u0026 Large Result Sets\n\n### Pagination Pattern\n```python\n# Retrieve all accounts in pages\nall_accounts = []\nfor page in client.get(\n    \"account\",\n    top=500,      # Records per page\n    page_size=500\n):\n    all_accounts.extend(page)\n    print(f\"Retrieved page with {len(page)} records\")\n\nprint(f\"Total: {len(all_accounts)} records\")\n```\n\n### Manual Paging with Continuation Tokens\n```python\n# For complex paging scenarios\nskip_count = 0\npage_size = 1000\n\nwhile True:\n    page = client.get(\"account\", top=page_size, skip=skip_count)\n    if not page:\n        break\n    \n    print(f\"Page {skip_count // page_size + 1}: {len(page)} records\")\n    skip_count += page_size\n```\n\n---\n\n## 9. File Operations\n\n### Upload Small Files (\u003c 128 MB)\n```python\nfrom pathlib import Path\n\nfile_path = Path(\"document.pdf\")\nrecord_id = \"account-guid\"\n\n# Single PATCH upload\nresponse = client.upload_file(\n    table_name=\"account\",\n    record_id=record_id,\n    file_column_name=\"new_documentfile\",\n    file_path=file_path\n)\nprint(f\"Upload successful: {response}\")\n```\n\n### Upload Large Files with Chunking\n```python\nfrom pathlib import Path\n\nfile_path = Path(\"large_video.mp4\")\nrecord_id = \"account-guid\"\n\n# SDK automatically chunks large files\nresponse = client.upload_file(\n    table_name=\"account\",\n    record_id=record_id,\n    file_column_name=\"new_videofile\",\n    file_path=file_path,\n    chunk_size=4 * 1024 * 1024  # 4 MB chunks\n)\nprint(f\"Chunked upload complete\")\n```\n\n---\n\n## 10. OData Filter Optimization\n\n### Case Sensitivity Rules\n```python\n# ❌ WRONG: Uppercase logical names\nresults = client.get(\"account\", filter=\"Name eq 'Contoso'\")\n\n# ✅ CORRECT: Lowercase logical names\nresults = client.get(\"account\", filter=\"name eq 'Contoso'\")\n\n# ✅ Values ARE case-sensitive when needed\nresults = client.get(\"account\", filter=\"name eq 'Contoso Ltd'\")\n```\n\n### Filter Expression Examples\n```python\n# Equality\nclient.get(\"account\", filter=\"name eq 'Contoso'\")\n\n# Greater than / Less than\nclient.get(\"account\", filter=\"creditlimit gt 50000\")\nclient.get(\"account\", filter=\"createdon lt 2024-01-01\")\n\n# String contains\nclient.get(\"account\", filter=\"contains(name, 'Ltd')\")\n\n# AND/OR operations\nclient.get(\"account\", filter=\"(name eq 'Contoso') and (creditlimit gt 50000)\")\nclient.get(\"account\", filter=\"(industrycode eq 1) or (industrycode eq 2)\")\n\n# NOT operation\nclient.get(\"account\", filter=\"not(statecode eq 1)\")\n```\n\n### Select \u0026 Expand\n```python\n# Select specific columns (improves performance)\nclient.get(\"account\", select=[\"name\", \"creditlimit\", \"telephone1\"])\n\n# Expand related records\nclient.get(\n    \"account\",\n    expand=[\"parentaccountid($select=name)\"],\n    select=[\"name\", \"parentaccountid\"]\n)\n```\n\n---\n\n## 11. Cache Management\n\n### Flushing Cache\n```python\n# Clear SDK internal cache after bulk operations\nclient.flush_cache()\n\n# Useful after:\n# - Metadata changes (table/column creation)\n# - Bulk deletes\n# - Metadata synchronization\n```\n\n---\n\n## 12. Performance Best Practices\n\n### Do's ✅\n1. **Use `select` parameter**: Only fetch needed columns\n   ```python\n   client.get(\"account\", select=[\"name\", \"creditlimit\"])\n   ```\n\n2. **Batch operations**: Create/update multiple records at once\n   ```python\n   ids = client.create(\"account\", [record1, record2, record3])\n   ```\n\n3. **Use paging**: Don't load all records at once\n   ```python\n   for page in client.get(\"account\", top=1000):\n       process_page(page)\n   ```\n\n4. **Reuse client instance**: Create once, use many times\n   ```python\n   client = DataverseClient(url, credential)  # Once\n   # Reuse throughout app\n   ```\n\n5. **Apply filters on server**: Let Dataverse filter before returning\n   ```python\n   client.get(\"account\", filter=\"creditlimit gt 50000\")\n   ```\n\n### Don'ts ❌\n1. **Don't fetch all columns**: Specify what you need\n   ```python\n   # Slow\n   client.get(\"account\")\n   ```\n\n2. **Don't create records in loops**: Batch them\n   ```python\n   # Slow\n   for record in records:\n       client.create(\"account\", record)\n   ```\n\n3. **Don't load all results at once**: Use pagination\n   ```python\n   # Slow\n   all_accounts = list(client.get(\"account\"))\n   ```\n\n4. **Don't create new clients repeatedly**: Reuse singleton\n   ```python\n   # Inefficient\n   for i in range(100):\n       client = DataverseClient(url, credential)\n   ```\n\n---\n\n## 13. Common Patterns Summary\n\n### Pattern: Upsert (Create or Update)\n```python\ndef upsert_account(name, data):\n    \"\"\"Create account or update if exists.\"\"\"\n    try:\n        # Try to find existing\n        results = list(client.get(\"account\", filter=f\"name eq '{name}'\"))\n        if results:\n            account_id = results[0]['accountid']\n            client.update(\"account\", account_id, data)\n            return account_id, \"updated\"\n        else:\n            ids = client.create(\"account\", {\"name\": name, **data})\n            return ids[0], \"created\"\n    except Exception as e:\n        print(f\"Upsert failed: {e}\")\n        raise\n```\n\n### Pattern: Bulk Operation with Error Recovery\n```python\ndef create_with_recovery(records):\n    \"\"\"Create records with per-record error tracking.\"\"\"\n    results = {\"success\": [], \"failed\": []}\n    \n    try:\n        ids = client.create(\"account\", records)\n        results[\"success\"] = ids\n    except Exception as e:\n        # If bulk fails, try individual records\n        for i, record in enumerate(records):\n            try:\n                ids = client.create(\"account\", record)\n                results[\"success\"].append(ids[0])\n            except Exception as e:\n                results[\"failed\"].append({\"index\": i, \"record\": record, \"error\": str(e)})\n    \n    return results\n```\n\n---\n\n## 14. Dependencies \u0026 Versions\n\n### Core Dependencies\n- **azure-identity** \u003e= 1.17.0 (Authentication)\n- **azure-core** \u003e= 1.30.2 (HTTP client)\n- **requests** \u003e= 2.32.0 (HTTP requests)\n- **Python** \u003e= 3.10\n\n### Optional Dependencies\n- **pandas** (Data manipulation)\n- **reportlab** (PDF generation for file examples)\n\n### Development Tools\n- **pytest** \u003e= 7.0.0 (Testing)\n- **black** \u003e= 23.0.0 (Code formatting)\n- **mypy** \u003e= 1.0.0 (Type checking)\n- **ruff** \u003e= 0.1.0 (Linting)\n\n---\n\n## 15. Troubleshooting Common Issues\n\n### ImportError: No module named 'PowerPlatform'\n```bash\n# Verify installation\npip show PowerPlatform-Dataverse-Client\n\n# Reinstall\npip install --upgrade PowerPlatform-Dataverse-Client\n\n# Check virtual environment is activated\nwhich python  # Should show venv path\n```\n\n### Authentication Failed\n```python\n# Verify credentials have Dataverse access\n# Try interactive auth first for testing\nfrom azure.identity import InteractiveBrowserCredential\ncredential = InteractiveBrowserCredential(\n    tenant_id=\"your-tenant-id\"  # Specify if multiple tenants\n)\n\n# Check org URL format\n# ✓ https://yourorg.crm.dynamics.com\n# ❌ https://yourorg.crm.dynamics.com/\n# ❌ https://yourorg.crm4.dynamics.com (regional)\n```\n\n### HTTP 429 Rate Limiting\n```python\n# Reduce request frequency\n# Implement exponential backoff (see Error Handling section)\n# Reduce page size\nclient.get(\"account\", top=500)  # Instead of 5000\n```\n\n### MetadataError: Table Not Found\n```python\n# Verify table exists (schema name is case-insensitive for existence, but case-sensitive for API)\ntables = client.list_tables()\nprint([t['table_schema_name'] for t in tables])\n\n# Use exact schema name\ntable_info = client.get_table_info(\"new_customprefixed_table\")\n```\n\n### SQL Query Not Enabled\n```python\n# query_sql() requires org config\n# If disabled, fallback to OData\ntry:\n    results = client.query_sql(\"SELECT * FROM account\")\nexcept Exception:\n    # Fallback to OData\n    results = client.get(\"account\")\n```\n\n---\n\n## Reference Links\n- [Official Repository](https://github.com/microsoft/PowerPlatform-DataverseClient-Python)\n- [PyPI Package](https://pypi.org/project/PowerPlatform-Dataverse-Client/)\n- [Azure Identity Documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme)\n- [Dataverse Web API Documentation](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/webapi/overview)\n","description":"Production-ready patterns and best practices extracted from Microsoft's official PowerPlatform-DataverseClient-Python repository, examples, and recommended workflows.","import":{"commit_sha":"541b7819d8c3545c6df122491af4fa1eae415779","imported_at":"2026-05-18T20:05:35Z","license_text":"MIT License\n\nCopyright GitHub, Inc.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.","owner":"github","repo":"github/awesome-copilot","source_url":"https://github.com/github/awesome-copilot/blob/541b7819d8c3545c6df122491af4fa1eae415779/instructions/dataverse-python-best-practices.instructions.md"},"manifest":{}},"content_hash":[61,34,92,163,147,224,202,216,80,39,68,54,90,15,47,187,233,198,111,118,36,84,204,73,169,232,48,251,12,127,169,129],"trust_level":"unsigned","yanked":false}