Analyze Images, Videos, Documents & Audio with Gemini Tools and Qwen LLM Agent

📁 Analyze uploaded images, videos, audio, and documents with specialized tools — powered by a lightweight language-only agent.

🧭 What It Does

This workflow enables multimodal file analysis using Google Gemini tools connected to a text-only LLM agent. Users can upload images, videos, audio files, or documents via a chat interface. The workflow will:

Upload each file to Google Gemini and obtain an accessible URL. Dynamically generate contextual prompts based on the file(s) and user message. Allow the agent to invoke Gemini tools for specific media types as needed. Return a concise, helpful response based on the analysis.

🚀 Use Cases

Customer support**: Let users upload screenshots, documents, or recordings and get helpful insights or summaries. Multimedia QA**: Review visual, audio, or video content for correctness or compliance. Educational agents**: Interpret content from PDFs, diagrams, or audio recordings on the fly. Low-cost multimodal assistants: Achieve multimodal functionality **without relying on large vision-language models.

🎯 Why This Architecture Matters

Unlike end-to-end multimodal LLMs (like Gemini 1.5 or GPT-4o), this template:

Uses a text-only LLM (Qwen 32B via Groq) for reasoning. Delegates media analysis to specialized Gemini tools.

✅ Advantages

| Feature | Benefit | | ----------------------- | --------------------------------------------------------------------- | | 🧩 Modular | LLM + Tools are decoupled; can update them independently | | 💸 Cost-Efficient | No need to pay for full multimodal models; only use tools when needed | | 🔧 Tool-based Reasoning | Agent invokes tools on demand, just like OpenAI’s Toolformer setup | | ⚡ Fast | Groq LLMs offer ultra-fast responses with low latency | | 📚 Memory | Includes context buffer for multi-turn chats (15 messages) |

🧪 How It Works

🔹 Input via Chat

Users submit a message and (optionally) files via the chatTrigger.

🔹 File Handling

If no files: prompt is passed directly to the agent. If files are included:

Files are split, uploaded to Gemini (to get public URLs). Metadata (name, type, URL) is collected and embedded into the prompt.

🔹 Prompt Construction

A new chatInput is dynamically generated:

User message

Media: [array of file data] 🔹 Agent Reasoning

The Langchain Agent receives:

The enriched prompt File URLs Memory context (15 turns) Access to 4 Gemini tools:

IMG: analyze image
VIDEO: analyze video
AUDIO: analyze audio
DOCUMENT: analyze document

The agent autonomously decides whether and how to use tools, then responds with concise output.

🧱 Nodes & Services

| Category | Node / Tool | Purpose | | --------------- | ---------------------------- | ------------------------------------- | | Chat Input | chatTrigger | User interface with file support | | File Processing | splitOut, splitInBatches | Process each uploaded file | | Upload | googleGemini | Uploads each file to Gemini, gets URL | | Metadata | set, aggregate | Builds structured file info | | AI Agent | Langchain Agent | Receives context + file data | | Tools | googleGeminiTool | Analyze media with Gemini | | LLM | lmChatGroq (Qwen 32B) | Text reasoning, high-speed | | Memory | memoryBufferWindow | Maintains session context |

⚙️ Setup Instructions

1. 🔑 Required Credentials

Groq API key** (for Qwen 32B model) Google Gemini API key** (Palm / Gemini 1.5 tools)

2. 🧩 Nodes That Need Setup

Replace existing credentials on:

Upload a file Each GeminiTool (IMG, VIDEO, AUDIO, DOCUMENT) lmChatGroq

3. ⚠️ File Size & Format Considerations

Some Gemini tools have file size or format restrictions. You may add validation nodes before uploading if needed.

🛠️ Optional Improvements

Add logging and error handling (e.g., for upload failures). Add MIME-type filtering to choose the right tool explicitly. Extend to include OCR or transcription services pre-analysis. Integrate with Slack, Telegram, or WhatsApp for chat delivery.

🧪 Example Use Case

> "Hola, ¿qué dice este PDF?"

Uploads a document → Agent routes it to Gemini DOCUMENT tool → Receives extracted content → LLM summarizes it in Spanish.

🧰 Tags

multimodal, agent, langchain, groq, gemini, image analysis, audio analysis, document parsing, video analysis, file uploader, chat assistant, LLM tools, memory, AI tools

📂 Files

This template is ready to use as-is in n8n. No external webhooks or integrations required.