Pure image processing pipeline.
Photon takes raw images and emits structured, AI-enriched data — vector embeddings, semantic tags, perceptual hashes, metadata, and LLM descriptions. Pure Rust, a single binary: no Python, no Docker, no cloud dependency. Input images, output JSON — and your backend owns everything downstream.
Every image is decoded exactly once. The raw bytes feed the BLAKE3 content hash and the pipeline in a single pass — validate, decode, hash, embed, tag.
SigLIP scores each image against 68,000+ terms in one matrix multiply. Vector embeddings, deduplicated semantic tags, and optional LLM descriptions fall out.
Structured JSON streams out — no database, no search engine, no server. Pipe it anywhere; your backend owns everything downstream.
Seven stages. Sixty-eight thousand terms. One matrix multiply. Every image read exactly once, every result emitted as JSON — no runtime to babysit.
Seven sequential stages, each independently testable, each individually skippable. Every image is read exactly once — the raw bytes feed both the BLAKE3 content hash and the decoder. CPU-heavy stages run inside async timeouts so a single corrupt file never blocks the pipeline. Stages transform progressively: raw pixels become structured vectors, semantic tags, and machine-readable metadata. No intermediate state. No side effects.
SigLIP scores every image against a vocabulary of 68,000+ terms in a single matrix-vector operation. The raw cosine similarity is calibrated through SigLIP's learned sigmoid scaling — constants derived from training, not heuristics. Hierarchy deduplication via WordNet hypernym chains suppresses redundant ancestors: when "labrador retriever" scores high, "dog" and "animal" are pruned automatically. The result is a precise, deduplicated tag set ranked by genuine visual confidence.
Not all 68,000 terms matter for every dataset. The relevance tracker organises vocabulary into three pools: Active terms are scored against every image. Warm terms are sampled periodically. Cold terms are dormant. Pool transitions happen during periodic sweeps — terms that consistently match get promoted, terms that never fire get demoted. When a term enters Active, its WordNet siblings are pulled into Warm for evaluation. The vocabulary adapts to the data.
No database. No search engine. No API server. No file watcher. Photon processes images and emits structured JSON — what happens downstream is your decision. Dual-stream output separates fast pipeline results from slow LLM enrichment. Pipe JSONL into any backend. Embed the core library as a Rust crate. Bring your own LLM key for descriptions — Ollama, Anthropic, OpenAI, or Hyperbolic. Single binary. 226 tests. MIT license.
Yes. Photon is open source under the MIT license, distributed via PyPI, GitHub Releases, and Cargo, with source at github.com/hejijunhao/photon.
Photon is written in pure Rust and ships as a single binary — no Python, no Docker, and no cloud dependency at runtime.
Photon is an image-processing pipeline that takes raw images and emits structured JSON: vector embeddings, semantic tags, perceptual hashes, metadata, and optional LLM descriptions.
Photon runs the SigLIP vision model on ONNX Runtime to score each image against a vocabulary of 68,000+ WordNet terms in a single matrix multiply, returning 768-dimensional embeddings and a deduplicated tag set.
No. Photon has no database, search engine, or API server; it processes images and emits structured JSON, and your backend owns everything downstream.
Photon supports Ollama, Anthropic, OpenAI, and Hyperbolic for optional image descriptions, using your own API key.