PrismaBuild
Keep the machines working on the right job.
A small lab has limited memory, shared disks and experiments that can take hours. PrismaBuild is the build and dispatch system that makes those resources usable together. It is a product in its own right, separate from the quantizer.
Declare the work. Keep the result.
A job names its code, inputs and resource needs. PrismaBuild sends it to a compatible worker when there is room. It records the outcome and stores successful results by content identity, so an unchanged job can reuse a verified result instead of running again.
Quantization jobs, tests and exports use this path. vLLM serving and work that runs vLLM are exempt from batch admission; their resource use still counts as outside load when PrismaBuild admits other jobs.
The content-addressed store (CAS) binds results to action keys. Git snapshots preserve the submitted code even if the working tree changes later. Check completion in the terminal record, logs and receipt.
Jobs travel through a pull queue that workers claim from.
The hardware has different jobs.
Sparky and Sparklina are GB10 / DGX Spark systems with 128 GB of unified memory each. The CPU and GPU share that memory. The dl380g10 is the x86 storage and CPU server, with about 300 GB of RAM. It holds the shared data and supplies the faster read tiers.
The storage links use NFS, a shared-file service, over RDMA, a network path designed for fast transfers between machines' memory.
How much memory jobs can use
Linux exposes 121.627 GiB on each Spark. PrismaBuild caps each Spark's aggregate memory offer at 104 GiB and dl380g10's at 96 GiB, adjusting admission to the available resources. The server's measured physical RAM is 294.523 GiB. Memory left outside a job budget supports the operating system, storage and other work.
Each Spark has one physical GPU.
Bring the next data closer before the job asks for it.
The hard-disk pool is the durable source. A dedicated SSD stage holds upcoming data, and a managed RAM window puts the next reads within reach of the fast network. The worker consumes that window while PrismaBuild prepares the next one.
The SSD stage and the RAM window hold part of a job's data at a time. Their reservations remain tied to the data while it is retained. Progress lets the system release consumed ranges and make room for what comes next.
A separate SSD acts as a re-read cache, called L2ARC. It caches previous reads while the SSD stage holds upcoming inputs.
Reservations and write barriers
The staged path is /stage/prewarm; the RAM policy names /ram/prewarm. Its configured default window is 160 GiB, with a 256 GiB ceiling, a 16 GiB system reserve and a 20 GiB ARC floor.
Movement and egress have their own action records. The tier coordinator composes the residency map; a consumer needs completed movement evidence and retained capacity. Produced outputs carry checksums and ownership. Handoffs and progress wait for committed output.
Write pacing is opt-in. Outputs can drain asynchronously over standard NFS as dependencies allow.
What has been demonstrated.
The deployed fleet has run real quantization, test and measurement jobs. The readiness record covers tests on every core host and GPU execution on both Sparks.
A separate storage experiment showed why memory placement matters: reading staged data from server RAM was much faster than reading it from the SSD.
Recorded checks
- The September 5 readiness record reports 1,835 passed and 3 skipped per host, plus 10/10 parallel test shards, for the recorded revision.
- On September 18, Sparky read a staged shard over a 100 Gbps NFS/RDMA link. With 16 concurrent direct-read streams of 256 MiB each, the measured rates were 2,402 MB/s from the SSD after an ARC miss and 10,045 MB/s from an ARC hit. Client page caching was bypassed (one shard, one link).
Sources: project status, dated readiness record and operating guide. This page is a dated snapshot.