← All projects
AffordRAG-Factory Edge AI Framework
A hierarchical edge VLM and knowledge graph framework that turns ambiguous shop-floor requests into explainable, executable multi-robot actions.
Edge-side embodied AI for high-mix manufacturing.
AffordRAG-Factory addresses manufacturing settings where workers describe problems by effect, not by exact robot command. The system combines local vision-language understanding with structured knowledge graph constraints so decisions remain interpretable and executable.
The framework targets low-compute, privacy-sensitive shop floors by using edge VLMs, retrieval-augmented affordance reasoning, robot capability checks, and a unified Skill-Code execution interface.
Submitted to ASME JCISE
The JCISE submission is an expanded version of our ASME IDETC/CIE 2026 conference paper.
Meng, Y., Yang, X., Frericks, J. B., Xie, J., De Austria, T., Singh, A., Favin, D., & Morkos, B. (2026). AffordRAG-Factory: A Hierarchical Edge VLM and Knowledge Graph Framework for Ambiguous Heterogeneous-Robot Task Execution in Manufacturing. Manuscript submitted to the ASME Journal of Computing and Information Science in Engineering (JCISE).
Results reported in the submission
Benchmark
Ambiguous instruction understanding
Evaluated on 119 ambiguous manufacturing instructions. Qwen3-VL-2B achieved 94.1% accuracy with an average response time of 897 ms. A pass required both the intended tool or semantic target and the expected Skill-Code output.
Robots
Heterogeneous robot deployment
Demonstrated on a Dobot E6 robotic arm for visual inspection, a Unitree G1 EDU humanoid for tool grasping and handoff, and a Unitree Go2 quadruped for navigation.
System
Local model deployment
Six local models from the SmolVLM and Qwen families were evaluated on an NVIDIA RTX 4070. All used less than 6 GB of GPU memory and exceeded 85% accuracy on the instruction benchmark.