Post
114
The capture stack for collecting high-fidelity egocentric RGB-D data on commodity mobile devices is now open-source.
https://x.com/fpv_labs/status/2082098617704436058
https://x.com/fpv_labs/status/2082098617704436058
AI & ML interests
None defined yet.
Recent Activity
View all activity
Post
1927
We just released an end-to-end system that sets a new state of the art in egocentric video understanding, generating fine-grained action labels from raw robot and human videos while outperforming Gemini, GPT, Claude, and other leading models.
https://x.com/fpv_labs/status/2079600323883331880
https://x.com/fpv_labs/status/2079600323883331880
Fix Gradio and dependency compatibility
#1 opened about 1 month ago
by 
Theivaprakasham
Fix Gradio and dependency compatibility
#1 opened about 1 month ago
by Theivaprakasham
Fix Gradio and dependency compatibility
#1 opened about 1 month ago
by Theivaprakasham
espejelomarΒ
posted an update 3 months ago
Post
4748
Sharing WorldForge with @abdelstark
It's an open-source Python project for evaluating and replaying robotics and world-model workflows.
The useful part is not only calling a model. WorldForge records the run, validates action shapes, translates outputs into actions, and keeps replay artifacts you can inspect later.
The current demo uses LeRobot + LeWorldModel on PushT through the official loader:
The harness also has replay-only paths for Cosmos-Policy and GR00T-style outputs, so you can inspect the provider contract from saved artifacts without keeping a GPU server online.
Try it:
Repo: https://github.com/AbdelStark/worldforge
Docs: https://abdelstark.github.io/worldforge/
Pre-1.0, MIT, and actively looking for contributors. Good areas:
- robotics provider adapters
- replay artifacts
- eval flows
- docs & first-run demos
Good first issues: https://github.com/AbdelStark/worldforge/contribute
If you're building robot policy evals or model adapters, would love a PR β or an issue describing what's missing.
It's an open-source Python project for evaluating and replaying robotics and world-model workflows.
The useful part is not only calling a model. WorldForge records the run, validates action shapes, translates outputs into actions, and keeps replay artifacts you can inspect later.
The current demo uses LeRobot + LeWorldModel on PushT through the official loader:
stable_worldmodel.policy.AutoCostModel("pusht/lewm")The harness also has replay-only paths for Cosmos-Policy and GR00T-style outputs, so you can inspect the provider contract from saved artifacts without keeping a GPU server online.
Try it:
pip install worldforge-aiuv run --extra harness worldforge-harness --flow robotics-compareRepo: https://github.com/AbdelStark/worldforge
Docs: https://abdelstark.github.io/worldforge/
Pre-1.0, MIT, and actively looking for contributors. Good areas:
- robotics provider adapters
- replay artifacts
- eval flows
- docs & first-run demos
Good first issues: https://github.com/AbdelStark/worldforge/contribute
If you're building robot policy evals or model adapters, would love a PR β or an issue describing what's missing.
satpalsrΒ
submitted a
paper to Daily Papers 3 months ago
satpalsrΒ
authored a
paper 3 months ago
Post
212
We're open-sourcing our infra with 10M+ frames of dataset!
We're releasing Stera, an open-source infra that turns an off-the-shelf device in your pocket into a high-fidelity multimodal data pipeline. It's built around four layers. Capture β Process β Evaluate β Export.
Stera Capture removes the need for bespoke/gated hardware and runs on an off-the-shelf iPhone. It fuses together synchronized RGB, IMU, Lidar-guided depth, and 6-DoF pose out of the box from ARKit and exports them to a raw MCAP file.
Dataset: fpvlabs/stera-10m
Launch Details: https://x.com/fpv_labs/status/2055262652033908832
We're releasing Stera, an open-source infra that turns an off-the-shelf device in your pocket into a high-fidelity multimodal data pipeline. It's built around four layers. Capture β Process β Evaluate β Export.
Stera Capture removes the need for bespoke/gated hardware and runs on an off-the-shelf iPhone. It fuses together synchronized RGB, IMU, Lidar-guided depth, and 6-DoF pose out of the box from ARKit and exports them to a raw MCAP file.
Dataset: fpvlabs/stera-10m
Launch Details: https://x.com/fpv_labs/status/2055262652033908832
Post
192
OpenAI is hiring for SLAM Engineers!
And open-source shouldn't lag behind.
It's pretty hard and necessary problem required to be solved for bringing generalisable robots in real-world.
We are pushing out first deep down & will be open-sourcing stuff in the next releases. Hope everyone is ready! Cheers to HF & more hugs.
Find us at https://x.com/fpv_labs/status/2042585804162371713
And open-source shouldn't lag behind.
It's pretty hard and necessary problem required to be solved for bringing generalisable robots in real-world.
We are pushing out first deep down & will be open-sourcing stuff in the next releases. Hope everyone is ready! Cheers to HF & more hugs.
Find us at https://x.com/fpv_labs/status/2042585804162371713
Post
14221
deepseek-ai/DeepSeek-OCR is out! π₯ my take ‡οΈ
> pretty insane it can parse and re-render charts in HTML
> it uses CLIP and SAM features concatenated, so better grounding
> very efficient per vision tokens/performance ratio
> covers 100 languages
> pretty insane it can parse and re-render charts in HTML
> it uses CLIP and SAM features concatenated, so better grounding
> very efficient per vision tokens/performance ratio
> covers 100 languages
- 5 replies
Post
7090
large AI labs open-sourced a ton of models last week π₯
here's few picks, find even more here merve/sep-16-releases-68d13ea4c547f02f95842f05 π€
> IBM released a new Docling model with 258M params based on Granite (A2.0) π ibm-granite/granite-docling-258M
> Xiaomi released 7B audio LM with base and instruct variants (MIT) XiaomiMiMo/mimo-audio-68cc7202692c27dae881cce0
> DecartAI released Lucy Edit, open Nano Banana π (NC) decart-ai/Lucy-Edit-Dev
> OpenGVLab released a family of agentic computer use models (3B/7B/32B) with the dataset π» OpenGVLab/scalecua-68c912cf56f7ff4c8e034003
> Meituan Longcat released thinking version of LongCat-Flash π meituan-longcat/LongCat-Flash-Thinking
here's few picks, find even more here merve/sep-16-releases-68d13ea4c547f02f95842f05 π€
> IBM released a new Docling model with 258M params based on Granite (A2.0) π ibm-granite/granite-docling-258M
> Xiaomi released 7B audio LM with base and instruct variants (MIT) XiaomiMiMo/mimo-audio-68cc7202692c27dae881cce0
> DecartAI released Lucy Edit, open Nano Banana π (NC) decart-ai/Lucy-Edit-Dev
> OpenGVLab released a family of agentic computer use models (3B/7B/32B) with the dataset π» OpenGVLab/scalecua-68c912cf56f7ff4c8e034003
> Meituan Longcat released thinking version of LongCat-Flash π meituan-longcat/LongCat-Flash-Thinking
- 2 replies
Post
3589
IBM just released small swiss army knife for the document models: granite-docling-258M on Hugging Face π₯
> not only a document converter but also can do document question answering, understand multiple languages π€―
> best part: released with Apache 2.0 license π use it with your commercial projects!
> it supports transformers, vLLM and MLX from the get-go! π€
> built on SigLIP2 & granite-165M
model: ibm-granite/granite-docling-258M
demo: ibm-granite/granite-docling-258m-demo π
> not only a document converter but also can do document question answering, understand multiple languages π€―
> best part: released with Apache 2.0 license π use it with your commercial projects!
> it supports transformers, vLLM and MLX from the get-go! π€
> built on SigLIP2 & granite-165M
model: ibm-granite/granite-docling-258M
demo: ibm-granite/granite-docling-258m-demo π
Post
1322
a ton of image/video generation models and LLMs from big labs π₯
> Meta released facebook/mobilellm-r1-68c4597b104fac45f28f448e, smol LLMs for on-device use π¬
> Tencent released tencent/SRPO, high res image generation model and tencent/POINTS-Reader, cutting edge OCR π
> ByteDance released bytedance-research/HuMo, video generation from any input β―οΈ
find more models, datasets, demos here merve/sep-11-releases-68c7dbfa26bea8cd921fa0ac
> Meta released facebook/mobilellm-r1-68c4597b104fac45f28f448e, smol LLMs for on-device use π¬
> Tencent released tencent/SRPO, high res image generation model and tencent/POINTS-Reader, cutting edge OCR π
> ByteDance released bytedance-research/HuMo, video generation from any input β―οΈ
find more models, datasets, demos here merve/sep-11-releases-68c7dbfa26bea8cd921fa0ac
Post
1105
fan-favorite vision LM Florence-2 is now officially supported in transformers π€
find all the models in
florence-community org π«‘
find all the models in
Post
1892
past week was great for open LLMs π₯ merve/sep-1-releases-68bede0e729c12597eefd050
> Google released google/embeddinggemma-300m, new embedding model with 300M params
> new update to Kimi-K2 just landed moonshotai/Kimi-K2-Instruct-0905 π
> OpenBMB released a new version to MiniCPM with 8B params openbmb/MiniCPM4.1-8B
also soooo many Qwen-Image & Kontext LoRAs dropped!
> Google released google/embeddinggemma-300m, new embedding model with 300M params
> new update to Kimi-K2 just landed moonshotai/Kimi-K2-Instruct-0905 π
> OpenBMB released a new version to MiniCPM with 8B params openbmb/MiniCPM4.1-8B
also soooo many Qwen-Image & Kontext LoRAs dropped!
Post
3786
upgrade your transformers π₯
it comes with insanely capable models like merve/sam2-66ac9deac6fca3bc5482fe30, microsoft/kosmos-2.5, and more π«‘
I built a notebook you can run with free Colab T4 to walk through the API for new models ππ»ββοΈ merve/smol-vision
fine-tuning will follow-up soon!
it comes with insanely capable models like merve/sam2-66ac9deac6fca3bc5482fe30, microsoft/kosmos-2.5, and more π«‘
I built a notebook you can run with free Colab T4 to walk through the API for new models ππ»ββοΈ merve/smol-vision
fine-tuning will follow-up soon!
Post
6344
large AI labs have dropped so many open models last week π₯ don't miss out on them
β Apple released on-device vision LMs apple/fastvlm-68ac97b9cd5cacefdd04872e & apple/mobileclip2-68ac947dcb035c54bcd20c47
β OpenGVLab released InternVL3.5, 32 new vision LMs with one based on gpt-oss! (OS) OpenGVLab/internvl35-68ac87bd52ebe953485927fb
β MSFT released a killer small TTS model (OS) microsoft/VibeVoice-1.5B
find more herehttps://huggingface.co/collections/merve/august-29-releases-68b5a3754cfb8abf59e2b486
β Apple released on-device vision LMs apple/fastvlm-68ac97b9cd5cacefdd04872e & apple/mobileclip2-68ac947dcb035c54bcd20c47
β OpenGVLab released InternVL3.5, 32 new vision LMs with one based on gpt-oss! (OS) OpenGVLab/internvl35-68ac87bd52ebe953485927fb
β MSFT released a killer small TTS model (OS) microsoft/VibeVoice-1.5B
find more herehttps://huggingface.co/collections/merve/august-29-releases-68b5a3754cfb8abf59e2b486
- 1 reply
Post
6129
first vision language model built off openai/gpt-oss-20b just dropped! π₯
InternVL3.5 comes with 32 models π€― pre-trained, fine-tuned, aligned in various sizes OpenGVLab/internvl35-68ac87bd52ebe953485927fb
comes with gpt-oss or Qwen3 for LLM part ‡οΈ
InternVL3.5 comes with 32 models π€― pre-trained, fine-tuned, aligned in various sizes OpenGVLab/internvl35-68ac87bd52ebe953485927fb
comes with gpt-oss or Qwen3 for LLM part ‡οΈ
- 1 reply
Post
3363
GPT-4.1-mini level model right in your iPhone π€―
openbmb/MiniCPM-V-4 is only 4B while surpassing GPT-4.1-mini in vision benchmarks π₯
allows commercial use as well!
openbmb/MiniCPM-V-4 is only 4B while surpassing GPT-4.1-mini in vision benchmarks π₯
allows commercial use as well!