AI News

Jinfer is an early-release AI inference engine designed specifically for the Java Virtual Machine. It supports chat, vision, audio, embeddings, reranking, and text-to-speech workloads while aiming to run without a Python runtime, ONNX, Docker containers, sidecar processes, or inter-process communication. Its JVM-oriented stack includes toknroll, a pure-Java tokenizer implementation; support for reading and writing GGUF and Safetensors model formats used by llama.cpp and Hugging Face; jam, which provides quantized matrix-multiplication kernels that the project describes as competitive with llama.cpp on CPUs; and jota, a tensor API targeting Java, CUDA, HIP, Metal, OpenCL, and Mojo. Jinfer also integrates with Spring AI and LangChain4j. GraalVM Native Image compatibility is intended to produce self-contained binaries with low overhead and millisecond startup. The current release is CPU-only, and its developers are seeking feedback, particularly on the API surface.