mlnewton
Writing Work Resume

Krea 2Matthew Newton

Krea 2 Krea 2
Krea 3 Krea 2 Other Research Just for Fun

Selected Work

Krea 3

Krea AI · Image editing

As of October 20, 2026, Krea 3 Ultra ranked #5 in Artificial Analysis image editing ↗ (1151 Elo, one point behind Nano Banana 2.1), #4 in Arena multi-image editing ↗ (preliminary), and #4 in Design Arena image editing ↗.

I contributed across Krea 3's full stack: pretraining, midtraining, post-training, prompt expansion, and the inference harness. I led data curation and built embedding-search infrastructure to efficiently index our image collection and mine editing pairs, using open-source VLMs to generate scores and captions. I also helped train VLM-based prompt expanders through SFT and RL.

I ran pretraining and post-training evaluations and ablations with teammates, and used midtraining data to teach what pretraining had missed. I led collection of human annotation data for DPO and reward model training, and contributed to the Fast and Ultra inference harnesses.

Krea 3 announcement ↗

Krea 2

Krea AI · Text-to-image generation

I worked across Krea 2's development on a small team, spanning pretraining, midtraining, post-training, and prompt expansion. My data work included curation and sampling pipelines to broaden concept coverage, and leading human annotation data collection for DPO and reward model training. The examples below highlight a few pieces of that work, from world knowledge to moodboard-driven generation.

Krea 2 reached the global top 10 on Artificial Analysis's text-to-image leaderboard ↗ (June 2026). More than 16 million cumulative Hugging Face downloads as of October 17, 2026, across its Raw ↗, Turbo ↗, and Comfy-Org ↗ repositories.

A few training contributions
  • Ran pretraining and post-training evaluations and ablations with teammates to find what actually helped.
  • Built the prompt expander from SFT through RL, designing text rewards to complement the team's image rewards.
    • Built moodboard-driven prompt expansion.Turned moodboards into text conditioning, selecting reference images for texture and style.
    • Added safety training to reduce unintended IP leakage and unwanted nudity while preserving accurate anatomy and artistic expression.
  • Owned data curation and midtraining concept coverage.Owned data curation and pipelines, using midtraining to close concept-coverage gaps, from entities to camera angles and styles.
  • Led collection of human annotation data for DPO and reward model training.

World Knowledge and Concept Coverage

I built the data pipelines to keep the long tail in the training mix. Embedding-based clustering broadened visual coverage; Wikipedia/Wikidata-guided sampling and caption searches helped retain named entities that ordinary cluster sampling could miss. Midtraining was our last chance to close gaps in entities, camera angles, and styles.

Krea 2 illustration of a baobab viewed from below, its branches spreading across a pink sky.
Baobab · fisheye perspective from below
Krea 2 illustration of a red tiger pouncing on a boar amid bright jungle foliage.
Tiger and boar · subjects in motion
Krea 2 ink-style illustration of a samurai drawing a sword, seen from a low, wide-angle viewpoint.
Samurai · pose and wide-angle composition

Selected generations from Krea's official examples ↗.

Moodboards and Prompt Expansion

I built and trained the text prompt expander behind moodboards. This is text-to-image generation: the moodboard's images are interpreted in text space, rather than passed to the image model as a set of direct image references. The expander folds their visual direction into the user's prompt, so a board can combine disparate references. A selected style-reference image separately supplies texture and style.

I trained the prompt expander through SFT and RL, designing text rewards to complement the team's image rewards. Safety training addressed unintended IP leakage and unwanted nudity while preserving accurate anatomy and artistic expression.

Below, the same two subject prompts run through four moodboards.

Cyber Zine outputs · same two prompts · click to expand

Krea 2 horse portrait in blue-and-black halftone collage with sharp radiating graphic shapes.
“Portrait of a horse”
Krea 2 chameleon in a sweater, rendered with pixelated collage textures and retro computer graphics.
“Chameleon wearing a sweater”

Minimalist Ink outputs · same two prompts · click to expand

Krea 2 horse portrait drawn with sparse black ink lines on white.
“Portrait of a horse”
Krea 2 chameleon wearing a sweater, drawn small in sparse black ink lines with broad white negative space.
“Chameleon wearing a sweater”

Futurist Glam outputs · same two prompts · click to expand

Krea 2 horse portrait with luminous metallic makeup and a silver mane against black.
“Portrait of a horse”
Krea 2 chameleon wearing sunglasses and an orange knit sweater, perched on a reflective metal chair.
“Chameleon wearing a sweater”

Thermal Airbrush outputs · same two prompts · click to expand

Krea 2 horse portrait with glowing pink, orange, and blue edges against a dark background.
“Portrait of a horse”
Krea 2 chameleon in a pale ribbed sweater, rendered in glowing pink, orange, and blue gradients.
“Chameleon wearing a sweater”

Krea 2 technical report (coauthor) ↗ · Explore Krea 2 ↗

Other Research

ARC Challenge — Discrete Diffusion–Based Reasoning

2025

I hacked Qwen3-8B to solve ARC puzzles by filling in the easy parts first, instead of working in typewriter order. I built the training and evaluation pipeline to compare the two approaches.

Technical write-up · Hacker News discussion ↗

Private Federated Learning and On-Device Recommendations

Google Gboard

Phones encode words into compact sketches; secure aggregation combines them before noise is added to release frequent-word counts. I built private heavy-hitter tasks and custom TensorFlow Lite ops to run SecAggIBLT efficiently on Android, helping improve typing models, emoji search, and sticker suggestions.

Selected contributions
  • Shipped the team's first personalized on-device models: sticker recommendations using item-item collaborative filtering and user-item embeddings.
  • Built private heavy-hitter tasks with IBLTs, secure aggregation, and differential privacy. Used the data to improve typing models, emoji search, and sticker suggestions.
  • Built custom TensorFlow ops to run SecAggIBLT efficiently on Android phones through TensorFlow Lite.
  • Helped establish federated model evaluation and resolved barriers to private heavy-hitter tasks across teams.

Drip.art — Personalization and Video Stylization

Co-founder and CTO · Later Comfy Org

I co-founded Drip.art and built its training pipelines and model-weight caching and hot-swapping engine, serving thousands of people’s fine-tunes from a handful of A100 GPUs.

Video stylization

Optical flow tracked a grid of points through the video. When uncovered area exceeded a threshold, we selected a new keyframe and reset the grid, with minimum spacing and a separate threshold for hard cuts.

We stylized keyframes together as a filmstrip in one ControlNet-conditioned diffusion pass, then split it back into frames. We filled the gaps with a few-shot neural model inspired by EbSynth, trained to map original keyframes to their stylized counterparts.

Personalization with DreamBooth

We fine-tuned Stable Diffusion on a person’s photos, binding their likeness to a prompt token. Prior preservation helped retain the base model’s general capabilities. I tuned the fine-tuning recipe and built the photo preparation, training, and model-serving infrastructure, including weight caching and hot-swapping between personalized models.

Just for Fun

Kinect the Dots

Interactive art · 2015

A wall of about 1,900 LEDs that turns nearby people into live RGB silhouettes. I built the installation from the physical structure and wiring to the C++ depth-camera processing and microcontroller code.

Shown at Bay Area Maker Faire, Burning Man, and Santa Cruz Glow. Received two Editor’s Choice awards at Maker Faire.

Inside the installation

A Kinect tracks people; a MacBook sends the display data over USB to a Teensy microcontroller driving the LEDs. Two mirrored acrylic layers create an infinity-mirror effect.

Maker Faire project ↗

Blaze/Bazel Build Widget

Hardware · 2017

Cold builds could take 30–45 minutes, and I kept missing when they finished—or failed. I built a USB-connected LED widget to make the result hard to miss, then taught teammates to assemble and program their own.

How it works

A Python daemon polls the build server and sends status changes to a Teensy microcontroller. A ring of LEDs pulses red for failure and green for success, inside a laser-cut wood enclosure with a white acrylic diffuser.

Experience

Krea AI

September 2025 - Current

Joined as the fourth AI researcher on the small team behind Krea 2 and Krea 3. Worked across pretraining, midtraining, post-training, and prompt expansion, and on Krea 3's inference harness. Led data curation for both models.

Birchwood Software Corp.

2024 - 2025

Co-founder. Built a general-purpose RAG engine with custom retrieval and ranking, delivered through Slack. Sold it to its main customer.

Comfy Org (formerly Drip.Art)

December 2022 - December 2024

Co-founder and CTO. Led a small engineering team and pre-seed fundraising, from personalized image generation to ComfyUI workflow infrastructure. Stayed on as an advisor after leaving the CTO role.

  • Built evaluations for personalized models and studied engagement across 10k+ users; brought ComfyUI's original author on as a co-founder and led the pivot to workflow infrastructure.
  • Built automated LoRA/DreamBooth training and served hundreds of custom models concurrently.
  • Built GPU orchestration across AWS, Lambda Labs, and CoreWeave. Delivered same-day fine-tuning and subsequent generation in under five minutes.

Meta

May 2022 - December 2022

Ads Core ML. Built and optimized AutoML systems; worked with other teams to measure the gains from different automated approaches.

Google

December 2016 - April 2022

Gboard — Federated Learning

July 2019 - April 2022

Worked on Gboard's modeling team and partnered across teams on private federated learning. Mentored teammates and gave tech talks on collaborative filtering.

Search — Frontend

December 2016 - July 2019

Improved mobile Search rendering through experiments; built libraries used by virtually every mobile query and async APIs. Supported shared components. Led Search Features Bootcamp sessions and managed two STEP interns whose work led to full-time offers.

Apple

May 2013 - December 2016

Built authoring and localization tools for the Tips app and interactive product tours in more than 20 languages. Led an exploratory support chatbot project, managed an intern, and mentored junior engineers.

Just For Fun

  • Technical blog: ML experiments and practical AI engineering, including how a simple document-reading agent outperformed our RAG pipeline. Read the blog.
  • Blaze/Bazel build widget (2017): Built an LED widget to flag finished or failed builds, then taught teammates to assemble and program their own.
  • Kinect the Dots (2015): Built an interactive installation with about 1,900 LEDs; received two Editor's Choice awards at the Bay Area Maker Faire ↗.

Education

Rochester Institute of Technology · 2009 - 2013
BS, Computer Science. Minors in Mathematics and Economics.