Case study / Research system / Personal
Synthetic Instinct
An automated trading research and execution system for crypto futures.
- Role
- Designed and built independently
- Period
- 2025 – present
- Status
- Private / Active research
- Type
- Research system / Personal
01
The problem
Automated trading systems fail in unglamorous ways: an order is sent twice after a timeout, a stale price is read as zero, a strategy that looked good in a backtest was really overfitted, or a process restarts without knowing what the exchange actually did.
Synthetic Instinct is an exercise in building that kind of system properly. It is a research and engineering system first; its value is in how decisions are made, executed, recorded and evaluated, not in any claim about returns.
02
Approach
The live process is a modular monolith with a clear decision pipeline: scan and hard gates, bounded predictive and scenario overlays, a unified candidate contract, a deterministic final rank, and execution through one gateway, followed by reconciliation of every trade and its outcome.
Narrow ports for the clock, market data, account and execution separate decision logic from the exchange, which makes deterministic replay of recorded events possible.
03
Architecture
- 1
Market data
Reads return a snapshot with a quality state, so missing evidence is explicit.
- 2
Decision
AI review is an advisory input; invalid data or a failed hard gate always blocks.
- 3
Risk & execution
The only permitted path for order mutations, with idempotent intents.
- 4
Evidence & operations
Append-only journal, health incidents, reconciliation and a read-only command center.
04
Key decisions
- D01
A single execution gateway
Every exchange mutation goes through one boundary. Simulated implementations are never used by live trading, and new abstractions cannot bypass the gateway.
- D02
Safety over availability after a crash
Order intents move through a durable ledger with exactly one claimant. An intent left in progress by a restart is never retried automatically, because the exchange may already have accepted it; it is surfaced for reconciliation instead.
- D03
Data quality is part of the type
Market and account reads carry VALID, DEGRADED or INVALID states. Degraded data cannot keep an automatic action, and invalid data always blocks.
- D04
AI as an advisor, not an authority
Multi-model reviews and scenario overlays are bounded inputs. Hard gates and the final ranking stay deterministic and testable.
- D05
Earn promotion with evidence
Changes move from shadow through canary, limited and expanded stages to primary, one approval at a time, each bound to fresh evidence.
05
The decision contract
Every candidate the scanner produces, swing or scalp, is reduced to the same record: a data-quality state, gates marked authoritative or advisory, expected return with a lower bound, downside (CVaR), execution cost, marginal portfolio risk and the action the strategy is asking for.
One pure function turns that record into an action tier and one sort ranks the whole set. Neither does any I/O, so the same inputs give the same answer in production, in replay and in tests. Try the rule below.
Data quality
Authoritative gates
Advisory gates
Strategy requests
Resolved action
AUTOMANUAL
Degraded data cannot keep an automatic action, so AUTO is reduced to manual review.
Reason codes
- DATA_DEGRADED
- advisory gate code
- Blocking is not negotiable
- Invalid data or any failed authoritative gate resolves to BLOCK, whatever the strategy requested.
- Degraded data loses automation
- A candidate built on degraded data can stay visible, but AUTO and MICRO_AUTO are reduced to manual review.
- Advisory means advisory
- Failed advisory gates are kept as reason codes for the operator. They can never act as a hard block.
- One rank, computed once
- Blocked candidates sort last; the rest by utility, falling back to the return's lower bound or expected return when utility is missing, with stable tie-breaks. Consumers read the stored rank instead of recomputing it.
06
Execution and recovery
Every order mutation goes through one gateway, and automated entries reach it as order intents. An intent's ID is derived from its signal, its content is fingerprinted with SHA-256, and it moves through a durable ledger that allows exactly one claimant.
The hard case is a crash or timeout after the request has left. The exchange may already have accepted the order, so the system does not guess: the intent is reported as unknown and left for reconciliation instead of being sent again.
- PENDING
- IN_PROGRESS
- ACCEPTED
- FILLED
- FAILED
- REJECTED
- UNKNOWN
One order, one terminal record.
- Stable client order IDs
- The gateway derives a stable exchange client order ID from each intent and, after an ambiguous response, looks the order up read-only before anything else is sent.
- Resting orders settle later
- A resting limit order is stored as ACCEPTED and moved to FILLED or FAILED by whichever path observes its outcome. Settled records never change again.
- Stops are replaced create-first
- A new stop is placed and verified at the full position size before the old one is cancelled. If the new stop cannot be verified, the old one stays.
- Protection is proven or unknown
- A position counts as protected only when current reduce-only stops on the closing side cover its full quantity. Anything weaker is reported as UNKNOWN, never as safe.
07
Ports and replay
Decision logic reaches the outside world through five narrow ports: clock, market data, account, execution and the decision journal. Live adapters wrap the exchange and the database; replay adapters are deterministic and need no network.
Market and account reads return a snapshot with a quality state, so missing or stale evidence can never turn into an unexplained zero. Live read adapters do not import order code, and simulated execution is never wired into live trading.
Run mode
snapshots carry VALID · DEGRADED · INVALID
Decision core
Same code in both modes
- Clock
- Market data
- Account
- Execution
- Decision journal
- Migrated behind golden masters
- Production ranking moved onto the replay-safe scan-cycle seam behind golden-master tests against the legacy ranker, covering visible, hidden, invalid and hard-blocked candidates.
- An append-only journal
- Decision events carry a correlation ID, an atomic per-correlation sequence, occurred and observed timestamps, and the build, policy and configuration that produced them. Duplicates are rejected by a unique index.
- Order-independent replay
- The replay runner drains a stable receive-time queue and hashes every input event, so shuffling the source records cannot change the trace or its final fingerprint.
08
From hypothesis to capital
A strategy idea starts as a written pre-registration: the causal thesis, one primary metric, acceptable risk, a frozen holdout and a parameter budget, all fixed before any test runs. Every parameter set tried is hashed into a trial ledger, and going over budget fails closed.
Evaluation uses purged walk-forward windows with an embargo, so no training label leaks into a test window, and only the variant selected across development windows ever touches the holdout. Results are discounted for the number of variants tried, and a rejected hypothesis is recorded as rejected rather than re-tuned until it passes.
Windows
- train
- purged labels
- embargo
- test
- W1traintest
- W2test
- W3test
- W4test
- HOdevelopment windowsdevelopment: every variant scored, selection on training dataholdout
- no capitalSHADOW
- CANARY
- LIMITED
- EXPANDED
- PRIMARY
risk ceiling rises per stage · one stage per approval
- Discounted for trying
- The overfitting report includes a deflated-Sharpe probability, seeded bootstrap bounds and a walk-forward probability of overfitting. A strong point estimate is not enough on its own.
- A gate with three answers
- The promotion gate returns PROMOTE_TO_CANARY, MORE_EVIDENCE or REJECT. Its blockers include orphan or duplicate orders, protection failures, stale decisions, reconciliation drift, shadow-replay disagreement and a missing rollback path.
- One stage at a time
- A policy moves from shadow through canary, limited and expanded to primary, one approved step at a time, each needing evidence newer than the last change. A drawdown or tail-risk regression rolls it back to shadow automatically.
09
Operating it
The system is run from two surfaces. A Telegram bot handles commands, alerts and confirmations. A web command center, built and deployed separately, gives a read-first view of positions, decisions, incidents, research and runtime health.
The web app holds no trading logic and no exchange client. The browser only talks to its own server, which calls the bot's operator API with a service credential and a verified operator session. The bot stays the authority for every decision and every order.
Request
unlinked backend → explicit unlinked state, never sample data
Browseroperator's session
Web serverNext.js, no trading code
Operator APIbot process
Read modelsaccount snapshot, journal, incidents
- Fails closed, never fills in
- When the console is not linked to an authenticated backend it shows an explicit unlinked state. It never substitutes sample financial data.
- One schema, both sides
- The operator API's Zod schemas generate an OpenAPI document and a typed client. Responses are validated again at runtime, and a drift check fails verification if the two diverge.
- Few, guarded actions
- The console's mutations (emergency halt, step-up resume, incident acknowledgement, single-order cancel, position close) are idempotent, and order and position actions are bound to the state the operator was looking at.
10
Verification and hardening
One command is the bar for every change. It runs type checks for production and test code, linting, architecture and dead-code audits, the API drift check, the full test suite and builds for the bot and the web app, then writes a report tied to the exact commit, branch, working-tree state and lockfile.
Recent work has gone into closing failure modes found by audits of the code and the live database, each change landing with regression tests.
- Missing risk data blocks entries
- A failed position read used to look like a halt that could then be lifted as "recovered". Missing data now blocks new entries with its own reason and never clears a stored halt.
- No gap when a stop moves
- Stop replacement now creates and verifies the new stop before cancelling the old one, closing a window in which a position had no stop.
- Webhook secret out of URLs
- Webhook requests are authenticated with a secret that never appears in a URL or access log, with an alert and automatic recovery if a mismatch ever locks the operator out.
- Every release traceable
- The image build refuses to run without a source revision, so each running release can be tied to a commit.
11
Tradeoffs
- Refusing to auto-reclaim interrupted orders occasionally needs manual reconciliation, accepted to rule out duplicate orders.
- A modular monolith keeps latency and operations simple; ports keep the boundaries testable.
- Conservative gates block more opportunities than a looser system would, by design.
12
Where it stands
- Production ranking, the full candidate trace and the scalp watchlist run through one deterministic, replay-safe scan-cycle seam.
- Deterministic event replay, purged walk-forward validation, pre-registered trials and calibration scoring for predictive signals.
- Operator tooling covering health incidents with escalation, execution quality and daily reconciliation against the exchange.
This is a personal research system, not a product or a financial service, and it makes no performance claims. The code is private.
Stack
- TypeScript
- Node.js
- Express.js
- MongoDB
- CCXT
- Exchange APIs
- OpenRouter
- Telegram Bot API
- Next.js
- Jest
- Docker
- Fly.io