Synthetic YOLO Training Data from Simulation - Auto-Labeled, No Camera Needed to Generate It.

A student research project (AI20K-170): PyBullet renders YCB objects at randomized angles, auto-generates YOLO bounding boxes, and optionally applies a diffusion realism pass. No real photos required to generate the dataset - real YCB photos are reserved for benchmark validation.

10 YCB objects Real household objects, ready to detect

500 frames/run 5 camera angles × 100 yaw steps

0 manual labels in the generated dataset YOLO bounding boxes generated automatically

What We Do

Turn Physics into Training Data

We simulate real-world household objects in PyBullet, render them from dozens of angles, then pass the frames through a generative AI realism pipeline - delivering YOLO-labeled image datasets ready for object detection training, without a single real photo needed for generation. Real YCB photos are used only to validate the benchmark.

⚙️

Physics Simulation

PyBullet renders 3D YCB objects under randomized lighting, backgrounds, and camera angles - producing hundreds of pixel-perfect training frames per run.

🎨

GenAI Realism Pass

An optional SD 1.5 + ControlNet diffusion pipeline transforms flat sim renders into photorealistic scenes, closing the sim-to-real gap that hurts model accuracy.

🏷️

Auto YOLO Labels

Every frame comes pre-labeled with tight bounding boxes and class IDs - no manual annotation, no LabelImg, no hours of clicking rectangles.

📦

One-Click Export

Download a ready-to-train ZIP with images, YOLO labels, depth maps, and a dataset report generated by an LLM - plug straight into Ultralytics YOLO.

Why This Project

Real Data Is Expensive. Synthetic Isn't.

Collecting and labeling real images for object detection costs weeks and thousands of dollars. We built this to prove that a well-designed synthetic pipeline - combined with generative AI - can produce datasets that train detectors as accurately as real photography, at a fraction of the cost.

No camera rig required

Anyone can generate a training dataset from a browser - no studio, no turntable, no photographer.

Infinite variation

Randomize lighting, backgrounds, materials, and camera angles in seconds - more data diversity than any real shoot can provide.

GenAI closes the realism gap

Diffusion-enhanced sim frames look like real photos, so models trained here generalize to real-world scenes rather than overfitting to synthetic rendering artifacts.

From research to demo in one tool

Built on the AI20K-170 thesis: GenAI synthetic data can beat sim-only baselines on standard YOLO benchmarks, now accessible to anyone with a browser.

YOLO 101

Never Heard of YOLO? Here's the 60-Second Version.

YOLO - You Only Look Once - is the most popular real-time object detection algorithm in the world. Give it an image, and it tells you: what objects are in it, where they are, and how confident it is - all in a single pass, fast enough to run on a webcam or a phone.

🖼️

Image In

You give YOLO a photo or video frame. It divides the image into a grid and examines every cell simultaneously - no sliding windows, no region proposals.

📐

Bounding Box Out

For each object found, YOLO outputs a bounding box - a rectangle that tightly wraps the object - plus a class label (e.g. "apple") and a confidence score.

⚡

Real-Time Speed

Modern YOLO (YOLOv8, YOLO11, YOLO26) runs at 100+ FPS on a GPU. That's what makes it practical for robotics, warehouse picking, medical imaging, and autonomous driving.

📝

How It Learns

YOLO is trained on labeled images - each image paired with a text file listing every object's class and box coordinates. That labeled data is exactly what this tool generates.

What a YOLO label file looks like

0  0.512  0.491  0.234  0.318
2  0.178  0.623  0.145  0.201

Column 1 - class ID (0 = apple, 2 = mug…)

Column 2–3 - box center X, Y (0–1 normalized)

Column 4–5 - box width, height (0–1 normalized)

One line per object. One .txt file per image. That's the entire YOLO label format.

Dataset Format

What's Inside Your ZIP - and Yes, It's YOLO-Ready.

The downloaded ZIP follows the standard Ultralytics YOLO directory layout exactly. Drop it into your training script - no renaming, no reformatting.

ZIP contents

          📦 your_dataset.zip

          ├── images/          ← training photos

          │   ├── frame_0000.png

          │   ├── frame_0001.png

          │   └── … (up to 500)

          ├── labels/          ← one .txt per image

          │   ├── frame_0000.txt

          │   ├── frame_0001.txt

          │   └── …

          ├── preview/

          │   └── preview_grid.jpg  ← visual check

          ├── metadata.json    ← sim config used

          ├── quality_report.json ← QA gate results

          └── generation_metadata.json

Ready to train with Ultralytics

          # 1. Unzip and add a data.yaml

          path: ./your_dataset

          train: images

          val: images  # split as needed

          nc: 10

          names: [apple, bleach_cleanser,

                   bowl, cracker_box,

                   master_chef_can, mug,

                   mustard_bottle, power_drill,

                   sugar_box, tomato_soup_can]

          # 2. Train

          yolo train model=yolo26n.pt data=data.yaml

✓

images/ + labels/ are paired by filename - YOLO finds the label for frame_0012.png by looking for frame_0012.txt in the sibling labels/ folder automatically.

✓

Coordinates are already normalized - YOLO expects values between 0 and 1, relative to image width/height. Our simulator outputs them that way, no conversion needed.

Benchmark · YOLO26n · Golden-2

Benchmark Results: Sim Dataset vs. Lab Photos

Three YOLO26n models trained under identical hyperparameters - one on real lab photos, one on domain-randomized sim renders, one on those renders after an SD 1.5 realism pass - evaluated on a balanced 240-image mixed test set (80 real + 80 sim + 80 realism, 24 per class across 10 YCB objects). Run date: 2026-06-28. The like-for-like comparison is realism vs sim (+0.024 mAP50-95 - same synthetic budget, only the realism pass differs); the larger gap vs lab_raw partly reflects how costly real-data collection becomes as objects are added. Results reflect this specific student thesis experiment.

lab_raw · baseline

0.762

mAP50

0.557

mAP50-95

real lab photos · 10 classes

sim · domain randomized

0.912

mAP50

0.826

mAP50-95

+0.269 vs lab_raw · sim renders

realism ✦ best overall

0.949

mAP50

0.850

mAP50-95

+0.024 vs sim · sim + SD 1.5

Per-class comparison: sim vs realism 10 classes · 240 test images (80 real + 80 sim + 80 realism)

Box P - Precision: of all boxes predicted, how many are correct R - Recall: of all real objects, how many were detected mAP50 - mean Average Precision at IoU 0.5: overall detection quality mAP50-95 - stricter metric across IoU 0.5–0.95: localization accuracy Δ - change from sim → realism (green = improved, red = regressed)

Class	Box P			R (Recall)			mAP50			mAP50-95
Class	sim	realism	Δ	sim	realism	Δ	sim	realism	Δ	sim	realism	Δ

Apple - sim vs generated (v4 halo-fix)

5 yaw angles · top row = PyBullet sim · bottom row = SD 1.5 realism pass

Magnify: Hover image to inspect

Move mouse over image to magnify · no scroll conflict

What's Next

The Road Ahead

This project doesn't stop at YCB. Here's where it's going.

Now

🧱

YCB Object Library

10 standard YCB objects. PyBullet physics simulation. SD 1.5 realism pass. YOLO auto-labeling pipeline.

📦

Custom 3D Object Import

Import any OBJ / GLTF model. Render at randomized poses. Compose into synthetic environments - not limited to lab objects.

Soon

🌍

Generative Scene Integration

Extract rendered objects into generative 3D environments (Isaac Lab / Cosmos-style). Richer background diversity, physically accurate lighting.

Future

⚡

Faster & Stronger Models

Upgrade to more capable diffusion models while reducing cost and inference time. Higher quality synthetic data at scale, accessible to more users.

The Team

👤

Phạm Hoàng Anh Kiệt

miniphak

Project Lead

👤

Nguyễn Tiến Sỉ

Member

👤

Nguyễn Trung Kiên

Member