ML · Data Eng2025
SynMedix AI
A distributed pipeline that processes roughly 10 GB of electronic health records at 3× the throughput of the serial baseline, with a generative layer on top producing 50,000+ synthetic patient records deployed on AWS SageMaker.
- Throughput over serial baseline
- 3×
- Synthetic records generated
- 50K+
- EHR corpus processed
- ~10 GB
- Real records reproduced
- 0
The problem
Electronic health records are simultaneously abundant and unusable. The volume is enormous, the privacy constraints are absolute, and serial processing is slow enough that iterating on a model means waiting overnight. Any team wanting to train on real clinical data hits all three walls at once — and the third wall is the one that quietly kills projects, because a research loop measured in days is a research loop nobody runs.
The approach
Two separate problems, solved in order. First the throughput: restructure ingest around parallel execution with optimised ETL stages so the corpus can be reprocessed in an afternoon rather than overnight. Then the privacy: train a generative model on the cleaned corpus that learns the joint distribution of the records rather than memorising any individual one, so downstream teams get a dataset they can experiment on freely.
The healthcare industry generates enormous amounts of EHR data but processing it at scale remains painfully slow. I built SynMedix to demonstrate that with the right distributed architecture, medical AI teams can iterate 3x faster on realistic, privacy-safe datasets.
Architecture
- 01
Ingest
Roughly 10 GB of raw EHR pulled into S3, partitioned so downstream stages can be worked in parallel rather than streamed serially.
- 02
Parallel ETL
Optimised transform stages executing concurrently — the change responsible for the 3× throughput improvement over the serial baseline.
- 03
Generative layer
PyTorch/TensorFlow models trained on the cleaned corpus to learn the joint distribution across clinical fields.
- 04
Serving
Deployed on AWS SageMaker for scalable inference, with DynamoDB and S3 backing metadata and generated artefacts.
Stack
- Modelling
- PyTorch · TensorFlow · Keras
- Pipeline
- Python · SQL · parallel ETL
- Cloud
- AWS SageMaker · S3 · DynamoDB
- Python
- SQL
- PyTorch
- TensorFlow
- Keras
- ETL
- AWS SageMaker
- AWS S3
- DynamoDB
Outcome
SynMedix emits synthetic patient records that preserve the statistical structure of the source corpus without carrying any individual through it. Downstream teams get a dataset they can train on and share freely — which was the entire point. The 3× throughput gain compounds: every experiment that used to cost a night now costs an afternoon.
Live demoNot a screenshot
Explore the latent space.
The generator described above, trained and exported to run client-side. Move a latent dimension or change the conditioning and the record regenerates — no server, no API call, no round trip.
Generator loads as you reach it…