Improve ML system performance for Modal by developing open-source projects and its container runtime to increase model throughput and reduce latency.
Scaling existing workAI / ML“If you are interested in contributing to open-source projects and Modal’s container runtime to push language and diffusion models towards higher throughput and lower latency, we’d love to hear from you!”
Product areasModal container runtime · Open-source ML performance projects · Language model inference · Diffusion model inference
GoalsIncrease throughput and reduce latency for language and diffusion models
TechPyTorch · vLLM · TensorRT · NVIDIA GPUs · CUDA · Linux kernel · Filesystems · Containers