Update edge function wrapper to emit backend metadata headers (X-Request-Id, X-Span-Id, X-Trace-Id, X-Duration-Ms) on responses; adjust logging to include duration and headers. Enhance edgeFunctionTracking to extract and propagate backend metadata from responses and errors; extend errorHandler to capture and log backend metadata for improved observability.
Improve CORS handling and error logging to fix moderation edge cases:
- Add x-idempotency-key to allowed CORS headers and expose explicit POST methods
- Extend CORS headers to include Access-Control-Allow-Methods
- Update edge function tracing and client error handling to better detect and log CORS/network issues
- Enhance error handling utilities to surface CORS-related failures and provide clearer user messages
Implements edge function, Django tasks, and UI hooks/panels for automatic retention of old metrics, anomalies, alerts, and incidents, plus updates to query keys and monitoring dashboard to reflect data-retention workflows.
Introduce statistical anomaly detection for metrics via edge function, hooks, and UI components. Adds detection algorithms (z-score, moving average, rate of change), anomaly storage, auto-alerts, and dashboard rendering of detected anomalies with run-once trigger and scheduling guidance.
- Add hooks and components for correlated alerts and incidents
- Integrate panels into MonitoringOverview
- Extend query keys for correlation and incidents
- Implement incident actions (create, acknowledge, resolve) and wiring
Fix security definer view issue by enabling security_invoker on grouped_alerts_view and implement grouped alerts system with new keys, hooks, components, and monitoring overview integration. Added migration to adjust view, updated query keys, created useGroupedAlerts, useAlertGroupActions, GroupedAlertsPanel, and updated MonitoringOverview to include grouped alerts.
Refactor to store park location data relationally:
- Retrieve park_submission_id from park_submissions
- Remove embedded temp_location_data usage
- Update park_submissions without location, then upsert location into park_submission_locations
- Align behavior with updateSubmissionItems pattern to fix JSON storage in Item history
Refactor src/lib/integrationTests/suites/moderationDependencyTests.ts to replace direct submission_items inserts with proper submission helper usage (submitParkCreation), aligning with relational schema. Update dep-001 to create parks via helpers and adjust item retrieval/cleanup accordingly. Modify dep-002 to verify relational submission structure via helpers, ensuring tests interact with new schema and avoid legacy item_data usage. This fixes schema/mapping mismatches and aligns tests with the approved Phase 2 approach.
Adds a new test data cleanup utility to safely remove test fixtures after integration test suites. Includes type-safe cleanup functions for parks, rides, companies, ride_models, locations, and submissions, with safety checks (is_test_data filters) and progress logging. Integrates cleanup invocation post-run to prevent database bloat and preserves safety against prod data deletion.
Add phase 2 migration for moderator INSERT policies and enhance test runner rate-limit mitigations:
- Introduce migration 20251110_fix_missing_insert_policies_phase2.sql to grant moderator INSERT capabilities for park_submission_locations, parks, rides, companies, ride_models, and locations with MFA checks.
- Update test runner to 8s base delays, preemptive cooldowns before heavy suites, and 18s post-suite delays for heavy suites, improving rate-limit handling.
Enhance retry logic to detect 429 rate limits, parse Retry-After headers, and apply smart backoff across all entity submissions. Adds rate-limit-aware backoff, preserves user feedback via UI events, and ensures retries respect server-provided guidance.
Add exponential backoff retry logic with user feedback
- Integrate with existing withRetry patterns
- Introduce unique retry IDs for submissions
- Emit consistent UI events for retry progress, success, and failure
- Enhance rate-limit handling including Retry-After scenarios
- Standardize baseDelay and shouldRetry across park, ride, company, ride_model, and other submissions
Implemented end-to-end fixes:
- Added INSERT RLS policies for park_submissions, ride_submissions, company_submissions, ride_model_submissions, and photo_submissions (own submissions and moderator access)
- Fixed process_approval_transaction to replace ride_type with category and updated references accordingly
- Enhanced rate limiting in testRunner.ts with a 6s base delay and 12s adaptive delays after submission-heavy suites
Adds comprehensive error pattern matching in formatTestError to detect and format Supabase constraint violations (RLs, not-null, unique, foreign keys, check constraints, and related schema/permission issues) into actionable messages, extracting relevant details for clearer test failure guidance. Includes handling for common error shapes and preserves fallback messaging.
Apply quick wins and pipeline fixes for integration tests:
- Remove is_test_data flag from location inserts
- Increase test delay from 2.5s to 5s and add delays between suites to curb rate limiting
- Replace [object Object] error formatting with formatTestError across 10 test suites (31 edits) and add import
- Refactor unit/conversion tests and performance tests to use the approval pipeline
- Extend edge function handling by ensuring item_ids are included in idempotency key inserts (edge function fix)
- Update auth, data integrity, edgeFunction, moderation, performance, submission, unitConversion, and versioning test files accordingly
Introduce visible countdown indicators for rate-limit delays between tests:
- Extend test runner to emit and expose delay progress updates.
- Update IntegrationTestRunner UI to display countdown while pausing.
- Ensure users see why tests are paused during delays.
Introduce automatic delay between test executions in the test runner to prevent rate limiting. Added a delayBetweenTests config (default 2.5s), a delay() helper, and updated runSuite to wait between tests. This helps avoid "Too many submissions in a short time" errors during test runs.
- Replace [object Object] errors with readable messages by using robust error formatting across test suites
- Introduce formatTestError helper and apply it in all catch blocks and error throws
- Update approvalPipelineTests and related suites to utilize improved error extraction for better debugging
Refactor versioningTests.ts, submissionTests.ts, and dataIntegrityTests.ts to replace direct DB inserts with the submission pipeline (submitParkCreation → approve → verify), aligning tests with RLS policies and validation flows.
Apply Phase 1 and 2 fixes:
- Remove display_name from location objects in approvalTestHelpers.ts
- Update submission_type to park/ride instead of park_create/ride_create
Add Markdown formatting utilities for test results, wire up clipboard copy in IntegrationTestRunner, and export new formatters. Introduce formatters.ts, extend index.ts exports, and implement copy all / copy failed / per-test copy functionality with updated UI.
Expand approvalPipelineTests with 12 tests: park/ride updates, composites, photo galleries, edge cases (locks, invalid refs, banned users), and versioning. Include helpers for photo submissions and direct entity creation, plus photo gallery support and invalid temp-ref handling.
Implement integration plan to connect to Lovable Cloud by adopting the Lovable Cloud toolchain flow:
- Initiate tool-based wiring for Lovable Cloud connectivity
- Set up scaffolding to enable approved workflow and data exchange with Lovable Cloud
- Prepare for secure auth/session handling and API interactions
- Outline groundwork for subsequent implementation steps in the Lovable Cloud integration path
Integrate Lovable Cloud tracing updates by enabling distributed tracing in edge functions, adjusting breadcrumb/trace propagation, and preparing RPC span handling. Files touched include edgeFunctionTracking and related RPC tracing scaffolding.
Implement distributed tracing across edge functions:
- Introduce span types and utilities (logger.ts enhancements)
- Replace request tracking with span-based tracing in approval and rejection flows
- Propagate and manage W3C trace context in edge tracking
- Add span visualization scaffolding (spanVisualizer.ts) and admin TraceViewer UI (TraceViewer.tsx)
- Create tracing-related type definitions and support files
- Prepare RPC call logging with span context and retries
Extend existing alert system to include real-time monitoring for rate limit violations and ban evasion attempts. This involves adding new reporting functions to `pipelineAlerts.ts`, integrating these functions into submission and company helper files, updating the admin dashboard component to display new alert types, and creating a database migration for the new alert type.
Implement rate limiting for `submitCompanyCreation` and `submitCompanyUpdate` to prevent abuse and ensure pipeline integrity. This includes adding checks for submission rate limits and recording submission attempts.
Added `recordSubmissionAttempt(userId)` to `submitParkCreation`, `submitParkUpdate`, `submitRideCreation`, and `submitRideUpdate` in `src/lib/entitySubmissionHelpers.ts`. This ensures that rate limit counters are incremented after a successful rate limit check, closing a vulnerability that allowed for unlimited submissions of parks and rides.
Implement Phase 4 by adding `recordSubmissionAttempt` and `withRetry` logic to the ban check for composite submissions. This ensures better error handling and prevents bypass of ban checks due to transient network issues.
Implement rate limiting, ban checks, retry logic, and breadcrumb tracking for ride model creation and update functions. Wrap existing ban checks and database operations in retry logic.
Implement rate limiting, ban checks, retry logic, and breadcrumb tracking for all 8 company submission functions: manufacturer, designer, operator, and property_owner (both create and update). This ensures consistency with other protected entity types and enhances the robustness of the submission pipeline.
Implement rate limiting, validation, retry logic, and ban checking for photo and timeline submissions. This includes updates to `UppyPhotoSubmissionUpload.tsx` and `entitySubmissionHelpers.ts`.
This commit implements Phase 3 of the Sacred Pipeline, focusing on enhanced error handling. It includes:
- **Transaction Status Polling Endpoint**: A new edge function `check-transaction-status` allows clients to poll the status of moderation transactions using idempotency keys.
- **Expanded Error Sanitizer Patterns**: The `src/lib/errorSanitizer.ts` file has been updated with more comprehensive patterns to remove sensitive information from error messages, making them safer for display and logging. User-friendly replacements for common errors are also included.
- **Rate Limiting for Submission Creation**: Client-side rate limiting has been implemented in `src/lib/submissionRateLimiter.ts` and applied to key submission functions within `src/lib/entitySubmissionHelpers.ts` (e.g., `submitParkCreation`, `submitRideCreation`, `submitParkUpdate`, `submitRideUpdate`) to prevent abuse and accidental duplicate submissions.
This commit implements Phase 4 of the Sacred Pipeline, focusing on transaction resilience. It introduces:
- **Timeout Detection & Recovery**: New utilities in `src/lib/timeoutDetection.ts` to detect, categorize (minor, moderate, critical), and provide recovery strategies for timeouts across various sources (fetch, Supabase, edge functions, database). Includes a `withTimeout` wrapper.
- **Lock Auto-Release**: Implemented in `src/lib/moderation/lockAutoRelease.ts` to automatically release submission locks on error, timeout, abandonment, or inactivity. Includes mechanisms for unload events and inactivity monitoring.
- **Idempotency Key Lifecycle Management**: A new module `src/lib/idempotencyLifecycle.ts` to track idempotency keys through their states (pending, processing, completed, failed, expired) using IndexedDB. Includes automatic cleanup of expired keys.
- **Enhanced Idempotency Helpers**: Updated `src/lib/idempotencyHelpers.ts` to integrate with the new lifecycle management, providing functions to generate, register, validate, and update the status of idempotency keys.
- **Transaction Resilience Hook**: A new hook `src/hooks/useTransactionResilience.ts` that combines timeout handling, lock auto-release, and idempotency key management for robust transaction execution.
- **Submission Queue Integration**: Updated `src/hooks/useSubmissionQueue.ts` to leverage the new submission queue and idempotency lifecycle functionalities.
- **Documentation**: Added `PHASE4_TRANSACTION_RESILIENCE.md` detailing the implemented features and their usage.
Approve and implement the Supabase migration for the pipeline monitoring alert system. This includes expanding alert types, adding new monitoring functions, and updating existing ones with escalating thresholds.
Implement comprehensive error handling and robustness measures across the entire pipeline as per the detailed plan. This includes database-level security, client-side validation, scheduled maintenance, and fallback mechanisms for edge function failures.
Correct parameter names in integration tests to resolve TypeScript errors. The errors indicate a mismatch between expected and actual parameter names (`p_user_id` vs `_user_id`) in Supabase-generated types, which are now being aligned.
Implement the destructive migration plan to remove the old approval flow entirely. This includes deleting the legacy edge function, removing the toggle component, simplifying frontend code, and updating documentation.
Update actions.ts and ApprovalTransactionToggle.tsx to default to the new atomic transaction RPC flow. The feature flag can now be used to disable the new flow for emergency rollback.
Centralize all business logic validation within the edge function for the submission pipeline. Remove validation logic from React hooks, retaining only basic UX validation (e.g., checking for empty fields). This ensures a single source of truth for validation, preventing inconsistencies between the frontend and backend.
Implement Phase 1 of the JSONB violation fix by creating the `park_submission_locations` table. This includes migrating existing data from `park_submissions.temp_location_data` and updating relevant code to read and write to the new relational table. The `temp_location_data` column will be dropped after data migration.
