Categories
AI

LLM and AI Coding Tools

AI and LLM Tool Ecosystem

A curated list and explanation of key elements in the AI and LLM ecosystem, including autonomous agents, coding assistants, frameworks, open-weight models, and self-hosted AI.

1. Autonomous / Agentic AI Platforms

Platforms that allow creation, orchestration, and deployment of AI agents that act autonomously or follow workflows:

  • CrewAI – Framework for orchestrating role-playing and autonomous AI agents
  • Autogen – Python framework for agentic AI. Best for orchestrating teams of AI agents.
  • AutoGPT – Create, deploy, and manage continuous AI agents for complex workflows. More autonomous.
  • LangChain – Flexible LLM workflows. Framework to convert LLMs into usable systems, orchestrate reasoning, tools, and action proposals. Focused on agents that call tools, apis and code.
  • dify – Open source self-hosted GUI based workflow / agent development
  • langflow – a drag‑and‑drop GUI / flow editor
  • haystack – AI orgastration framework – focused on RAG (retrieval augmented generation) adn document understanding and retrieval

2. IDE / Coding Assistants

2.1 VSCode Plugins

  • Continue – Build and run custom agents across IDE, terminal, and CI

2.2 CLI Agents

Open-Source

  • Opencode – Terminal-based AI coding agent that can use any AI model
  • Aider – Terminal coding assistant

Proprietary / Requires Login

Note: Requires account login or subscription

  • Claude Code – AI‑powered coding assistant/agent from Anthropic
  • OpenAI Codex – Converts natural language into code

3. Web Interfaces / UIs

  • OpenWeb-UI – User-friendly web interface for interacting with AI models

4. Vector Databases

Vector databases are used to store and search large numbers of vector embeddings (numerical representations of text, images, etc.) efficiently. It allows for searching for meaning not just keywords.

5. Frameworks and Tools

  • GPTCache – Caching layer for LLMs; speeds up requests using semantic caching
  • promptfoo – tool that helps you test, compare, and evaluate LLM prompts and models automatically. UNittests for prompts.
  • prompttools – Another prompt testing tool
  • mem0 – Adds long term memory to AI systems – so they don’t forget across sessions.

6. Open-Weight Models

Open-weight models have publicly available weights that allow local inference. Full training code and data may not be open. These models can be run as local LLMs, sometimes with cloud offload options.

Examples of Open-Weight Models:

  • Qwen 3
  • gpt-oss-20b
  • DeepSeek r1
  • GLM 4.5
  • Gemma (Google DeepMind)
  • Qwen2.5-coder:1.5b-base is a standalone code LLM — it can generate code but cannot act as a tool-calling agent

Note: Full-sized open-weight models are ~6 months behind top closed models in intelligence, but still slower at the same scale.
Source: blog.brokk.ai

6.1 Local LLM Runners / Inference Runtimes

Tools for running open-weight models locally or in hybrid mode:

  • Ollama
  • LM Studio
  • Hugging Face Hub + CLI – find, share and use pre-trained AI models and datasets
  • vllm – open-source high-throughput, memory-efficient inference and serving engine for large language models (LLMs)
  • gpt4all – Run LLMs locally on low spec hardware

Many of these tools can also offload inference to cloud models when local hardware cannot handle large weights.

7. Paid / Hosted Services

AI Routing & Gateways

  • openrouter.ai – Unified gateway for AI services; allows using different LLMs without integrating each one manually

Free Tier:

  • ~50 requests per model marked as :free daily

8. Self-Hosted / Local-First AI

Platforms for running AI locally, as a drop-in replacement for hosted APIs:

  • LocalAI – Open-source alternative to OpenAI, Claude, etc.; OpenAI-compatible local API and broader AI capabilities (text, audio, images, embeddings) with flexible deployment.

7. Data connectors

  • llama_index – bridge between data and LLMs
  • private-gpt – ask questions to your documents privately. ie. upload documents and ask questions.

8. More Lists

9. Rankings and Stats

10. Real World AI Usage

  • Generate Brand Logos – Instead of using Fiverrr…brands can be created using this tool. Selling at 7 times the price of what a fiverr person would offer.