Columbia EECS · Summer 2026 · Interview prep · Vol. 1 of 2
Kostic project
deep-dive — all twelve subprojects, analyzed
A project-by-project read of AIDL's open summer slots, scored against a real
record in hardware-systems and applied AI. Where the fit is honest, where the
crowd will cluster, and what to walk into the Zoom with.
Subject
Prof. Zoran Kostic AIDL Lab · Columbia EE
Interview
Wed 2026-05-27 3:00pm EDT · Zoom
Projects analyzed
11 subprojects + 1 standalone (AI on the EDGE)
Primary target
AI on the EDGE Secondary: Distillation SAM→YOLO
Companion to kostic-research-projects.md
DOC · KP · A1 · 2026-05
Kostic dossier ·Vol. 1
Section 00 · How to read this
00How to read this
A six-section read, ranked by fit against a real record.
Fit is scored against the actual portfolio — a 2-way out-of-order RISC-V core
with a 27.5% geomean cycle cut, the DE1-SoC VGA pipeline plus Linux kernel
driver, ESP32 / UAV work under real-time constraints, Palimpsest NYC's
agentic loop at 75.5% citation correctness, and the Gemma rank-1 ablation
/ distillation study. Where a project doesn't touch those, the score says so.
Real-time 3D is the hook; you lack 3D-vision background
7
Surgery — Agentic Vision
Surgery
MEDIUM
MEDIUM
Agentic + edge latency angle; domain unfamiliar
8
Surgery — Video World Models
Surgery
LOW-MED
MEDIUM
A4 edge-deploy is your only hook; diffusion-heavy
9
PCatcher (plastics)
Environment
LOW
LOW
Jetson/DeepStream is the only overlap
10
Boundless water-trash (UE5)
Environment
LOW
LOW
Pure UE5 synthetic-data; not your stack
11
Microplastics monitoring
Environment
LOW
LOW
Spectral imaging + environmental; weak match
11 distinct subprojects in the doc; "Surgery" umbrella adds the parent framing for 12 entries total.
AI on the EDGE is a standalone project, not a subproject.
Kostic dossier ·Vol. 1
Section 01 · The lab
01The lab
Who Kostic is, and how his lab actually works.
Twenty years in industry R&D before academia — Bell Labs, AT&T Research,
Thomson, MathWorks, Broadcom. Roughly 36 patents. Deep roots in SoC and
heterogeneous/parallel computing. The kind of advisor who notices when a
candidate can speak about cycles and cache lines, not just losses.
TitleProfessor of Professional Practice, EE
DirectorEE MS Program
AffiliationData Science Institute
PhDEE · University of Rochester
Emailzk2172@columbia.edu
Siteaidl.ee.columbia.edu
ScholarTlPI8yIAAAAJ
The one-sentence lab thesis
Efficient applied AI with a focus on inference — making models run in
real time on real hardware. That industry / systems lineage is why hardware
depth is the rare, valuable half he doesn't see often.
The four-course training ladder
Every project draws on one or more. Course pages live on the AIDL site.
AI on the EDGE, distillation, all edge-deploy tracks
Funding & testbeds
Name-drop sparingly to show you did homework — never as filler.
$26M
NSF CS3 Engineering Research Center · Smart Streetscapes
$3.8M
COSMOS³ supplement · PAWR wireless+edge testbed, West Harlem
$1.2M
NSF CPS · urban digital-twin
25k
A100 GPU-hours from NVIDIA grant + 2× Jetson AGX Orin
Also: NOAA microplastics grant · NIH R01 (speech/health).
The deliverables regime — know this going in
Cadence
Daily & weekly rhythm
Daily activity log
Daily GitHub commits
Weekly oral report
1–2 meetings/week (in-person full-time in summer)
End of term
Publish-and-ship
Conference-style engineering report
~24-slide deck
Oral presentation
Clean GitHub with READMEs
The caveat that sets your odds
His own doc: “It is very rare that a first-semester student is accepted,”
and it's “desirable to take one of Prof. Kostic's courses before applying.”
You're finishing your first semester with no Kostic course — a marginal case on
paper. Your job in the interview is to make the hardware-systems differentiator
outweigh that. (More in §5.)
Kostic dossier ·Vol. 1
Section 02 · Project-by-project
02Project-by-project analysis
Eleven subprojects, four tiers — scored against a real portfolio.
Each card carries the project's umbrella, fit, predicted crowd, the
Kostic-backed papers / courses it draws on, a sketch of what a 12–14 week
summer would look like, and an honest read on career compounding.
Tier 1Strongest fit · his core thesis · hardware-systems differentiator carries weight
1
Standalone · primary target
AI on the EDGE — Optimization of Inference for DL on Constrained GPU / IoT Devices
★
Fit · HighCrowd · HighCourse · E6692Devices · Jetson · Coral · NCSUp to 3 creditsLimited paid
What it is
Optimize inference for deep AI on networked low-power portable devices.
Partition LLMs and other DL models into distributed modules that fit
limited memory and compute; share load with edge-cloud compute and data
servers; measure and minimize latency; explore real-time
constraints. Hands-on with NVIDIA Jetson Nano, Google Coral, Intel Neural Compute
Stick. Apps in smart cities and healthcare.
Kostic-background link
Course
EECS E6692 Deep Learning on the Edge is literally this project
in course form (Jetson labs, profiling, compression, edge↔cloud connectivity).
Paper · Distributed VLMs (PerCom 2025 Workshop)
Vision encoder on edge, language generation on cloud, +33% throughput
vs cloud-only. This is "partition models across edge and cloud."
Paper · SASS — Streetscape Application Services Stack
The NVIDIA grant (Jetson AGX Orin + A100 hours) explicitly funds
low-latency multi-sensor edge fusion without cloud dependency — this
project's money.
What a summer looks like (≈12–14 weeks)
Wk 1–2
Set up a Jetson Orin; reproduce the Distributed VLMs split as a
baseline; profile where latency / energy actually goes (profiling instinct shines).
Wk 3–6
Pick a model (an urban detector or a small VLM) and implement
a partitioning scheme; instrument an end-to-end latency budget (SLA) across
edge↔cloud.
Wk 7–10
Core contribution — e.g. a smarter split point under a latency
SLA, or quantization / operator-scheduling on the edge half; measure latency /
energy / accuracy Pareto.
Wk 11–14
Deploy on real COSMOS-style data, write the conference-style
report, build the ~24 slides, present.
Kostic dossier ·Vol. 1
Section 02 · Tier 1 continued
Fit vs. you — HIGH
His doc's wording ("partition LLMs into distributed modules," "share load with
edge-cloud," "minimize latency," Jetson / Coral / NCS) is almost a transcript of
your cold-email pitch. Assets line up cleanly:
DE1-SoC co-design + kernel driver → comfort at the HW / SW boundary
and with memory-mapped data movement, which is exactly where edge-inference
latency hides.
OoO RISC-V (27.5% cycle reduction) → you reason about pipelines,
utilization, and where cycles go — rare among ML-only applicants.
Gemma distillation / ablation → credible on the model-compression half.
Coral / NCS → his doc names them, so the earlier mention is on-target
(the lab's published work leans Jetson; frame Coral / NCS as "I've also
worked with…").
Predicted popularity — HIGH
Edge ML is hot and the title is magnetic. But most applicants are ML-only; the
differentiator is hardware-systems depth, which is your moat. You're competing on
a crowded field but from an unusual angle.
Career value
Straight line into edge-ML / MLSys / inference-optimization roles
(the most hardware-adjacent, fastest-growing ML niche) and into hardware-aware ML
research. Compounds your existing "I build the silicon under the code" identity
better than anything else on this list.
2
Environment umbrella · strong secondary
Distillation for Low-Complexity AI Models ★
Fit · HighCrowd · Med-HighCourses · E6691 + E6692Device · JetsonScope · open ("TODO add details")
What it is
Distill the SAM model down to YOLO models for real-time object
detection. Optimize compute and memory. Deploy on NVIDIA Jetson.
The doc literally says "TODO add details" — so the scope is yours to help define,
which is an opportunity, not a red flag.
Kostic-background link
Course
E6691 Advanced DL (distillation, model compression, YOLO evolution, SAM /
foundation models) + E6692 (edge deployment).
Paper · Monocular 3D Tooltip Tracking (MDPI Electronics 2025)
Uses exactly this recipe — Florence2 + SAM2 zero-shot, refined
with YOLOv11 supervised training (84.5% → 91.2% Jaccard). The
SAM→YOLO distillation pattern is already live in the lab.
Related
The Constellation / Boundless detection work is the data side of the same coin.
What a summer looks like
Wk 1–2
Stand up SAM / SAM2 as teacher and a YOLO student; reproduce
a baseline detection pipeline on Jetson; measure the teacher's latency (slow)
vs target.
Wk 3–6
Implement knowledge distillation — match the YOLO student to
SAM's segmentation / detection on an urban or environmental dataset.
Wk 7–10
Push compute / memory optimization (quantization, pruning) to
hit a real-time frame-rate budget on Orin; quantify accuracy lost vs latency
gained.
This is the cleanest scoped, most "ownable in one summer" project for you. You
already have distillation experience (the Gemma project), it ends in a hard latency
number on real hardware (your comfort zone), and the open "TODO" scope lets you
propose the exact contribution. Lower competition than AI on the EDGE because it
sits under the less-glamorous "Environment" heading.
Predicted popularity & career value
Med-High. Distillation is trendy, but it's buried under
"Environment," so casual applicants skip it. Good for you: real fit, thinner
field. Same edge-ML / model-compression track as #1, with a concrete distillation
artifact you can show. Strong portfolio piece for MLSys roles and PhD apps.
Kostic dossier ·Vol. 1
Section 02 · Tier 2 — solid fit
Tier 2Solid fit · agentic loop transfers from Palimpsest · lighter HW lever
Real vehicle / pedestrian trajectories from the COSMOS testbed. Replicate
trajectory-prediction models for comparison; integrate traffic-light state +
environmental constraints into the simulator; integrate LLM / VLM into
high-level decision making.
Kostic-background link
Paper
Data-Driven Traffic Simulation for an Intersection in a Metropolis
(CVPR 2024 Workshop) — two-stage trajectory generation (coarse sampling →
TrajNet++ refinement), 0.36 FDE @ 20 FPS on A100.
Paper
Language-Guided Traffic Simulation and Rare Event Synthesis (CVPR 2026
Workshop) — natural-language control over scenarios, LLM-to-loss, rare-event
synth.
"Integrate LLM / VLM into high-level decision making" maps onto your Palimpsest
agentic-loop work. Pure trajectory modeling isn't your background, but the LLM /
agentic layer is, and there's a modest HW / real-time lever (20 FPS target). Less
hardware than Tier 1. Crowd: high — self-driving + LLM is a magnet
topic; much of the competition will be CV-native students with more trajectory
experience than you. Bridges toward autonomous-systems / CV research and
agentic-AI roles; good but less unique to your HW identity than Tier 1.
4
Smart Streetscapes
Real-Time Agentic Models for Smart Streetscapes
Fit · Med-HighCrowd · Very HighScale · 1000s of cameras · real-timeCourse · E6691
What it is
Implement agentic AI workflows for streetscapes (cf. Gemini
"agentic vision"). Detect objects + weather in street scenes; find limitations in
existing agentic workflows and improve them; run locally over 1000s of
cameras in real time. Doc flags it as fast-moving / frontier.
Fit · popularity · career
Your Palimpsest project is a bounded two-tool agentic
loop with honest pre-registered evaluation — directly transferable to "agent loop
with bounded latency, find + fix limitations." The "real-time over 1000s of
cameras" angle even invites your hardware instinct. Weaker on the CV-perception
side. Crowd: very high — "agentic" is the single hottest keyword
in the doc; this will draw the most applicants. Strong fit but the most
crowded — your Palimpsest artifact is what separates you from the pack here.
Agentic AI is a fast-rising hiring area; combined with the real-time / edge
framing, it's a strong story for applied-AI and research roles.
5
Smart Streetscapes
Adaptive Data Collection and Annotation
Fit · MediumCrowd · MediumTrack · theory-leaningPaper · ICML 2025
What it is
Refine a theoretical framework for optimizing data collection
for robust model training under multi-distribution scenarios; larger-scale
experiments on classification, detection, segmentation, VLM. Uses COSMOS
multi-view feeds. Skills: DNN training; LLM/VLM finetuning a plus;
ML theory a plus.
Kostic-background link
Paper: Adaptive Data Collection for Robust Learning Across Multiple
Distributions (ICML 2025) — UCB sampling + online gradient descent,
minimax-regret guarantees. Course E4040 + ML-theory foundations.
Fit · popularity · career
Your Mathematics of Deep Learning (A) course and the Gemma
rank-1 ablation show you can handle the theory / perturbation framing, but this is
the least hardware project in the set and leans toward students who want
an ML-theory trajectory. Honest read: doable, not your sweet spot.
Theory-flavored projects self-select a smaller, more specialized applicant pool.
Best for a theory / ML-research or PhD path; weaker for your hardware-systems
identity.
Kostic dossier ·Vol. 1
Section 02 · Tier 3 — lighter fit
Tier 3Lighter fit · edge-deployment is the only hook · domain unfamiliar
6
Smart Streetscapes
3D Models for Dynamic Streetscapes
Frontier 3D-vision models (SAM3D, Hunyuan3D, TRELLIS); combine 3D
reconstruction with background in dynamic urban scenes; run in real
time in complex streetscapes; tackle depth estimation / Gaussian
splatting and inpainting (LAMA too low-quality, FLUX Kontext insufficient) to
remove dynamic objects.
Fit · popularity · career
Only real hook is "run these in real time" — your optimization
/ HW instinct. You have no 3D-vision / Gaussian-splatting background, so you'd be
climbing a learning curve on the core. Pitch only if you're excited to
learn 3D vision. Cool but specialized; needs 3D-vision chops, which thins
the field. 3D vision / graphics-ML — a different track from yours; broadens but
doesn't compound.
7
Surgery
Agentic Vision Models
Fit · MediumCrowd · MediumDomain · medical · Lenox HillHook · agentic + edge latency
What it is
Build an agentic vision system that constructs structured
surgical context from live video (segmentation / detection tools) and reasons
over it in real time, under latency constraints. Tasks: agent loop with bounded
latency; perception tools (seg / detection / pose / depth); context construction +
temporal memory; latency-constrained edge deployment.
Kostic-background link
Course E6691 (agents, foundation models); paper Monocular 3D Tooltip
Tracking (Florence2 + SAM2 + YOLOv11) and the IPAL / Lenox Hill surgical
collaboration.
Fit · popularity · career
Same Palimpsest-agentic + edge-latency transfer as #4, plus the explicit "deploy
on edge under latency constraints" hook. But the surgical domain is unfamiliar and
access likely depends on the Lenox Hill collaboration. Niche domain (medical)
narrows the pool despite the agentic hotness. Respectable, domain-specific.
8
Surgery
Video Models in Robot-Assisted Surgery (World Models)
Fit · Low-MedCrowd · MediumTrack A3/A4 only · edge accel + Orin / Thor
What it is
Train / deploy video generation / world models predicting future
surgical scenes in real time from live robotic camera input; low-latency on edge.
Track A: video diffusion (Wan 2.2, LTX-Video 2), conditioned
prediction, inference acceleration (reduced-step diffusion, LoRA, distill + compile),
edge deploy on AGX Orin and AGX Thor (A4). Track B: kinematics +
world models from depth / pose.
Fit · popularity · career
Hook is narrow but real: A3 (distill + compile, reduced-step diffusion)
and A4 (edge deployment + latency / FPS measurement) sit right in your
wheelhouse. But the bulk is video-diffusion training — not your background
— and the surgical domain is unfamiliar. Pitch only the acceleration / deployment
tracks if you raise this one. Diffusion + world models is trendy; surgical domain
narrows it. Generative-video / world-models is a frontier area; the edge-acceleration
slice connects to your strengths, the rest doesn't.
Kostic dossier ·Vol. 1
Section 02 · Tier 4 — honest weak fit
Tier 4Honest weak fit · don't pitch these
Project 09 · Environment
PCatcher — AI-based plastics collection on water surface
Fit · LowCrowd · Low
Train object-detection models for water-surface obstacle detection (RGB and / or
LiDAR); integrate NVIDIA DeepStream SDK on Jetson embedded devices.
Single overlap: Jetson / DeepStream embedded deployment, which
your embedded background touches. Everything else (marine object detection, the
physical collection device) is outside your record. Career value limited for your
direction.
Project 10 · Environment
Boundless — Water-Surface Trash Detection via UE5
Fit · LowCrowd · Low
Use Unreal Engine 5 to generate a synthetic RGB dataset for water-surface
trash detection (HQ RGB + depth + bounding boxes). One student.
Backing paper: Boundless (UE5 synthetic data, +7.8 mAP vs CARLA) +
Constellation dataset (13K images). This is a UE5 / synthetic-data
engineering role. No game-engine / 3D-asset pipeline experience in your
record; nothing maps here. Off-track for career.
Project 11 · Environment
Microplastics — AI Real-Time Monitoring
Fit · LowCrowd · Low
Distinguish plastics from natural debris via spectral + color-camera
imaging for selective waste collection; tested in Hudson / Bronx Rivers.
NOAA-funded; Lamont / Climate-School collaboration.
Spectral imaging + environmental sensing; outside your background. Only thread
is "real-time," which is thin here. Mention environmental interest only
if asked; don't pitch.
Kostic dossier ·Vol. 1
Section 03 · Cross-project comparison
03Cross-project comparison
All eleven projects on five axes, side by side.
#
Subproject
Fit
Crowd
Paid / Credit
HW lever
1-summer ownable
Career compound
1
AI on the EDGE
High
High
limited paid / ≤3 cr
High
Yes
Best
2
Distillation SAM→YOLO
High
Med-High
limited paid / ≤3 cr
High
Yes — cleanest
High
3
Trajectory Prediction
Med-High
High
≤3 cr
Low-Med
Yes
Med
4
Agentic Streetscapes
Med-High
Very High
≤3 cr
Med
Partly (frontier)
High
5
Adaptive Data Collection
Med
Med
≤3 cr
Low
Partly (theory)
Med (theory track)
6
3D Models
Med
Med
≤3 cr
Med (real-time)
Risky
Med (new track)
7
Surgery — Agentic Vision
Med
Med
≤3 cr
Med
Partly
Med (medical)
8
Surgery — Video World Models
Low-Med
Med
≤3 cr
Med (A3/A4)
No
Med
9
PCatcher
Low
Low
≤3 cr
Low-Med
—
Low
10
Boundless
Low
Low
≤3 cr
Low
—
Low
11
Microplastics
Low
Low
≤3 cr
Low
—
Low
Where the crowd will cluster
Popularity synthesis
Agentic + edge + LLM/VLM draws everyone
The applicant crowd will cluster on the agentic + edge + LLM/VLM
labels: #4 (Agentic Streetscapes) draws the most, then
#1 (AI on the EDGE) and #3 (Trajectory).
Environment (#9–11) and the theory project (#5) draw the fewest. Surgery (#7–8)
is gated by domain interest and collaborator access.
What that means for you
Be the rare hardware-systems person on a crowded ML project
Your edge isn't being one more ML applicant on the hot projects — it's being
the rare hardware-systems person on a project where everyone
else is ML-only. That argues for #1 (AI on the EDGE) as primary
and #2 (Distillation) as the high-fit, lower-competition fallback.
Let Kostic steer, but go in anchored on #1 with #2 as the graceful pivot.
Kostic dossier ·Vol. 1
Section 04 · Career mapping
04Career-development mapping
Which projects compound toward which careers.
Project
Skills you'd build
Compounds toward
AI on the EDGE
Edge inference profiling, model partitioning, latency / energy Pareto, Jetson / CUDA
Edge-ML / MLSys / inference-optimization roles; hardware-aware ML research; PhD in efficient ML
Distillation SAM→YOLO
Knowledge distillation, quantization / pruning, real-time deployment, a shippable artifact
Model-compression / MLSys; strong portfolio + paper for PhD apps
3D reconstruction, Gaussian splatting, real-time vision
Graphics-ML / 3D vision (new track)
Surgery (both)
Diffusion / world models OR agentic vision + medical domain
Medical-AI; generative video
Environment (×3)
Detection + Jetson / DeepStream deployment
Applied embedded CV (narrow)
The compounding read
#1 and #2 are the only two that deepen your existing "I build the
silicon under the code" identity rather than starting a new track. #4 (agentic)
is the best non-hardware bet because Palimpsest already gives you a head start
and the market for agentic skills is rising fast.
Kostic dossier ·Vol. 1
Section 05 · Interview playbook
05Interview playbook · Wed 5/27, 3:00pm Zoom
Tomorrow. Lead with the hardware-systems line.
Self-contained. The general strategy + Seok-vs-Kostic decision frame live in the brief.
My background is the hardware side of edge inference — I can make models actually run fast on the device, not just train them.
That's his lab's whole thesis. Say it early.
Two concrete pitches
Lead with the first.
Pitch · 01 · Primary
Edge–cloud partitioning of an urban VLM / detector under a latency SLA on Jetson Orin
Explicitly extending the Distributed VLMs result (vision encoder on
edge, language on cloud). Offer to profile where the latency really goes and
find a better split point.
Pitch · 02 · Graceful pivot
Distillation-vs-partitioning tradeoff study
Distill SAM → YOLO to a real-time edge student model and quantify accuracy /
latency / energy against the partitioning approach. This is the cleanest
one-summer deliverable if he wants something self-contained.
Papers to name-drop — one specific thing each
Distributed VLMs (PerCom 2025)
"The +33% throughput from running the vision encoder on edge is exactly the
partitioning angle I want to push on."
Monocular 3D Tooltip Tracking
"You already use the SAM2 → YOLOv11 distillation pattern; I'd love to apply
that to an edge latency budget."
SASS — Streetscape Application Services Stack
"The >10× edge-compute throughput result is the kind of systems win I
care about."
Smart questions to ask him
Is the summer edge work closer to the Distributed VLMs partitioning
line or to a distillation / optimization track?
Which device — Jetson AGX Orin? Is Coral / NCS in play, or Jetson-only in
practice?
Is the target deliverable a workshop paper + public code release?
Given I'm finishing my first semester, is this a realistic summer for me, or
would you rather I take E6692 first and start in the fall?
On registration / funding: would this be ENGI E4900 (zero-credit) or ELEN
E6001/2, and is it paid, credit, or volunteer?
Kostic dossier ·Vol. 1
Section 05 · Landmines & pre-call
Honesty landmines — have an answer ready
Landmine · 01
First-semester acceptance "very rare" + no Kostic course
Acknowledge it; counter with the hardware-systems differentiator and your
shipped projects.
Landmine · 02
Coral / NCS mentioned in cold email
His doc lists them, so you're not off-base; just note Jetson is the house
platform.
Landmine · 03
Deliverables regime
Show you welcome the daily-log / weekly-report / final-paper cadence. You
already ship public GitHub + portfolio sites; say so.
Landmine · 04
International / on-campus-only constraint
Summer must be a Columbia-registered role; maps cleanly to ENGI E4900 / ELEN
E6001-2. Frame it as "I'm set up to register either way," not a complication.
30-second pre-call checklist
2:55 PM Wed
Zoom link arrived? (he sends just before — check inbox at ~2:50pm)
Have open: this doc, CV link, the 4 portfolio URLs (riscv-p6,
palimpsest-nyc, geometry-of-alignment, pvzfpga)
One sentence ready on each of: RISC-V core, DE1-SoC, Palimpsest, Gemma
distillation
No collision with Seok's Wed meeting (you kept 3:00–4:00pm clear for
Kostic)