# Ollama

Ollama is a platform for running and building with open models locally or in the cloud, with a REST API and Python and JavaScript libraries.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Open by Default, Already Occupied: Threat Actors are Renting your Compute

DevFeed: [Open by Default, Already Occupied: Threat Actors are Renting your Compute](<https://devfeed.tech/articles/open-by-default-already-occupied-threat-actors-are-renting-your-compute-141103.md>)

Original publisher: [Read original article](<https://www.rapid7.com/blog/post/tr-self-hosted-ai-renting-compute-exposed-servers>)

Author: Christiaan Beek

Published: 2026-11-18T14:38:55Z

Content type: article

Language: en

Sources: [Rapid7 Cybersecurity](<https://devfeed.tech/sources/rapid7-cybersecurity-blog.md>)

Topics: [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Threat monitoring](<https://devfeed.tech/topics/threat-monitoring.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [OSINT](<https://devfeed.tech/topics/osint.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [22](<https://devfeed.tech/tags/22.md>), [abliterated-models](<https://devfeed.tech/tags/abliterated-models.md>), [about](<https://devfeed.tech/tags/about.md>), [account](<https://devfeed.tech/tags/account.md>), [after](<https://devfeed.tech/tags/after.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [alert](<https://devfeed.tech/tags/alert.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [compute](<https://devfeed.tech/tags/compute.md>), [conf](<https://devfeed.tech/tags/conf.md>), [cybersecurity-company](<https://devfeed.tech/tags/cybersecurity-company.md>), [exposure-assessment-platform](<https://devfeed.tech/tags/exposure-assessment-platform.md>), [exposure-management](<https://devfeed.tech/tags/exposure-management.md>), [internet](<https://devfeed.tech/tags/internet.md>), [managed-detection-and-response](<https://devfeed.tech/tags/managed-detection-and-response.md>), [managed-security-solutions](<https://devfeed.tech/tags/managed-security-solutions.md>), [more](<https://devfeed.tech/tags/more.md>), [open](<https://devfeed.tech/tags/open.md>), [other](<https://devfeed.tech/tags/other.md>), [people](<https://devfeed.tech/tags/people.md>), [self-hosted](<https://devfeed.tech/tags/self-hosted.md>), [shows](<https://devfeed.tech/tags/shows.md>), [snapshot](<https://devfeed.tech/tags/snapshot.md>), [threat](<https://devfeed.tech/tags/threat.md>), [threat-research](<https://devfeed.tech/tags/threat-research.md>), [vulnerability-management](<https://devfeed.tech/tags/vulnerability-management.md>)

### AI overview

A scan of 20,354 preselected internet-facing hosts found 7,714 unauthenticated AI or notebook API findings on 7,627 hosts, mostly Ollama. After filtering 429 likely deception hosts, the reported critical findings fell from 387 to 39, underscoring that exposure counts need qualification. Read-only Ollama model lists revealed apparent third-party activity: callback-style registry pulls, server IPs and markers in model names, and encoded paths targeting sensitive files. These artifacts indicate attempted access, not proof of successful exploitation; the scan sent no inference, write or exploit requests, and could not establish whether targets remained vulnerable. The article recommends inventorying public and personally expensed cloud/VPS systems, treating suspicious artifacts as compromise, rotating credentials and rebuilding, binding runtimes privately behind authentication, restricting egress, patching, logging state-changing API calls, and assigning each server an owner. The findings come from one snapshot of preselected IPv4 addresses and model-name indicators; clean lists do not establish a clean host.

### Source excerpt

A snapshot of 20,354 internet-facing hosts shows self-hosted AI deployed the way we deployed databases a decade ago. The more interesting part is what other people had already left inside.

## Graph-RAG for Enterprise Documents: Building a Knowledge Graph from PDFs, Wikis, and Databases

DevFeed: [Graph-RAG for Enterprise Documents: Building a Knowledge Graph from PDFs, Wikis, and Databases](<https://devfeed.tech/articles/graph-rag-for-enterprise-documents-building-a-knowledge-graph-from-pdfs-wikis-and-databases-140616.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/graph-rag-for-enterprise-documents-building-a-knowledge-graph-from-pdfs-wikis-and-databases/>)

Author: Shittu Olumide

Published: 2026-11-18T11:00:00Z

Content type: tutorial

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Neo4j](<https://devfeed.tech/topics/neo4j.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Tutorial](<https://devfeed.tech/topics/tutorial.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistant](<https://devfeed.tech/tags/ai-assistant.md>), [api](<https://devfeed.tech/tags/api.md>), [arangodb](<https://devfeed.tech/tags/arangodb.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [cmdb](<https://devfeed.tech/tags/cmdb.md>), [configuration-management-database](<https://devfeed.tech/tags/configuration-management-database.md>), [confluence](<https://devfeed.tech/tags/confluence.md>), [connection-pool](<https://devfeed.tech/tags/connection-pool.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [graph](<https://devfeed.tech/tags/graph.md>), [graph-rag](<https://devfeed.tech/tags/graph-rag.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [pdfs](<https://devfeed.tech/tags/pdfs.md>), [python](<https://devfeed.tech/tags/python.md>), [rag](<https://devfeed.tech/tags/rag.md>), [wikis](<https://devfeed.tech/tags/wikis.md>)

### AI overview

The excerpt describes a three-tier Graph-RAG design that prioritizes live service-registry facts over wiki runbooks and PDF context. A small SPOC quad store holds registry and wiki facts with source labels; SQLite is used in the example, with Postgres presented as an isolated swap. Runbook prose is parsed with key-value regexes, while PDFs are extracted with pdfplumber, split into roughly 500-character chunks, and embedded in Chroma as a fallback. All tiers are queried and fused in one prompt: conflicting lower-priority facts are excluded, and PDF context can supplement structured facts. The example uses spaCy plus a known-service-name list and runs locally with Ollama and llama3.2. Production considerations include keeping registry data live, refreshing wiki/PDF indexes regularly, adapting extraction to real wiki formats, applying access controls before prompting, and using a graph database for larger connected datasets. The approach avoids a routing model but adds token cost and depends on the model following explicit rules.

### Source excerpt

Prerequisites You'll need Python 3.

## Serving Scikit-LLM Pipelines as REST APIs with FastAPI

DevFeed: [Serving Scikit-LLM Pipelines as REST APIs with FastAPI](<https://devfeed.tech/articles/serving-scikit-llm-pipelines-as-rest-apis-with-fastapi-135526.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/serving-scikit-llm-pipelines-as-rest-apis-with-fastapi/>)

Author: Iván Palomares Carrascosa

Published: 2026-11-16T11:00:00Z

Content type: tutorial

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [FastAPI](<https://devfeed.tech/topics/fastapi.md>), [text-classification](<https://devfeed.tech/topics/text-classification.md>), [REST API](<https://devfeed.tech/topics/rest-api.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [classification](<https://devfeed.tech/tags/classification.md>), [fastapi](<https://devfeed.tech/tags/fastapi.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [local](<https://devfeed.tech/tags/local.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [rest](<https://devfeed.tech/tags/rest.md>), [rest-api](<https://devfeed.tech/tags/rest-api.md>), [rest-apis](<https://devfeed.tech/tags/rest-apis.md>), [serve](<https://devfeed.tech/tags/serve.md>), [serving](<https://devfeed.tech/tags/serving.md>), [text](<https://devfeed.tech/tags/text.md>), [text-classification](<https://devfeed.tech/tags/text-classification.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

The excerpt describes serving a Scikit-LLM zero-shot text classifier locally through FastAPI and Ollama. It configures a local Ollama model, builds a classifier with scikit-learn's fit/predict pattern, serializes it to a joblib file, then loads it in an asynchronous FastAPI app with a /predict endpoint that can accept multiple texts. The app is run with uvicorn and tested through FastAPI's interactive docs at /docs. The label set supplied during fit defines the only categories the classifier can return; the LLM's weights are not updated, so inputs are mapped into that closed set. The excerpt presents this as a simple one-stage pipeline and notes that Scikit-LLM can support larger pipelines with preprocessing.

### Source excerpt

In this article, you will learn how to serve a Scikit-LLM zero-shot text classification pipeline, powered by a local Ollama model, as a REST API...

## How to detect shadow AI: unapproved model APIs and local LLMs

DevFeed: [How to detect shadow AI: unapproved model APIs and local LLMs](<https://devfeed.tech/articles/how-to-detect-shadow-ai-unapproved-model-apis-and-local-llms-134658.md>)

Original publisher: [Read original article](<https://nirmata.com/2026/11/16/how-to-detect-shadow-ai/>)

Author: Ritesh Patel

Published: 2026-11-16T05:03:28.562711Z

Content type: tutorial

Language: en

Sources: [Nirmata](<https://devfeed.tech/sources/nirmata.md>)

Topics: [shadow AI](<https://devfeed.tech/topics/shadow-ai.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [API Governance](<https://devfeed.tech/topics/api-governance.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [cluster-security](<https://devfeed.tech/tags/cluster-security.md>), [container](<https://devfeed.tech/tags/container.md>), [egress](<https://devfeed.tech/tags/egress.md>), [flag](<https://devfeed.tech/tags/flag.md>), [images](<https://devfeed.tech/tags/images.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local](<https://devfeed.tech/tags/local.md>), [local-llms](<https://devfeed.tech/tags/local-llms.md>), [logs](<https://devfeed.tech/tags/logs.md>), [model](<https://devfeed.tech/tags/model.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [pod](<https://devfeed.tech/tags/pod.md>), [ps](<https://devfeed.tech/tags/ps.md>), [security](<https://devfeed.tech/tags/security.md>), [servers](<https://devfeed.tech/tags/servers.md>), [shadow-ai](<https://devfeed.tech/tags/shadow-ai.md>), [tags](<https://devfeed.tech/tags/tags.md>)

### AI overview

Shadow AI detection needs three complementary checks because each sees different activity: inventory Kubernetes pod and init-container images, probe reachable Ollama servers, and review controlled egress logs. The image queries identify candidate model-server images, but tags or digests must be compared with the approved list; the standard container query misses init containers. Ollama's documented GET endpoints distinguish API availability (/api/version), locally available models (/api/tags), and models loaded in memory (/api/ps), so an empty process list does not rule out an installed idle model. Egress logs can reveal hosted API calls that inventory and probes miss; findings should be grouped by source, and laptops require endpoint telemetry or network logs. Pulls can add models after image approval. A hit needs owner review: the checks show activity, not whether it was approved. The excerpt says AIControls supports specified Claude paths and per-user tokens, but does not claim it detects unapproved endpoints or local models.

### Source excerpt

Detect shadow AI in Kubernetes: scan pod images for model servers, probe Ollama's /api/tags and /api/ps, and flag unapproved model API calls in egress logs.

## Choosing Quantized Code LLMs by File Size and Energy per Solved Task

DevFeed: [Choosing Quantized Code LLMs by File Size and Energy per Solved Task](<https://devfeed.tech/articles/choosing-quantized-code-llms-by-file-size-and-energy-per-solved-task-135805.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17605>)

Author: Aditi Patodiya

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [code](<https://devfeed.tech/tags/code.md>), [code-assistant](<https://devfeed.tech/tags/code-assistant.md>), [coder](<https://devfeed.tech/tags/coder.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [energy](<https://devfeed.tech/tags/energy.md>), [fanless](<https://devfeed.tech/tags/fanless.md>), [file](<https://devfeed.tech/tags/file.md>), [files](<https://devfeed.tech/tags/files.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [gguf](<https://devfeed.tech/tags/gguf.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [inference](<https://devfeed.tech/tags/inference.md>), [laptop](<https://devfeed.tech/tags/laptop.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local](<https://devfeed.tech/tags/local.md>), [local-inference](<https://devfeed.tech/tags/local-inference.md>), [size](<https://devfeed.tech/tags/size.md>), [solved](<https://devfeed.tech/tags/solved.md>)

### AI overview

On a fanless MacBook Air M2, tests of 24 Qwen2.5-Coder GGUF files across 542 EvalPlus tasks found that the larger-model-at-lower-precision rule did not pay at the 3 GB file-size tier. The guide selects 1.5B-q4_K_M and 3B-q4_K_M at 1 and 2 GB; both were on the correctness and energy frontier under every accounting method. The only measured 7B file fitting the 3 GB tier, 7B-q2_K, showed no significant solve gain over the 2 GB pick, while using 2.03 times the energy per solved task on CPU, GPU and memory rails (1.66-1.81 times on whole-system power). At 8 GB, 7B-q6_K did yield a significant gain over the 2 GB pick, at higher energy cost. From 1.5B upward, q4_K_M was within 3 percentage points of the highest measured precision at its size (90% CI).

### Source excerpt

arXiv:2611.17605v1 Announce Type: cross Abstract: A developer running a code assistant on a laptop downloads one model file, and local inference moves the energy of every request onto that machine. A common rule says a larger model at lower precision tends to beat a smaller one at higher precision. We ran 24 GGUF files of Qwen2.5-Coder at four sizes on a fanless MacBook Air M2 and priced each in energy per solved task on 542 EvalPlus tasks. The result is a selection guide indexed by Ollama's listed file size, with an evidence label on each pick. At 1 and 2 GB it picks the q4_K_M files of the 1.5B and 3B models, both on the correctness and energy frontier under every accounting. At 3 GB as listed, the larger-model rule picks 7B-q2_K, the only measured 7B file that fits, and there it does not pay. That file brings no significant gain in solves over the 2 GB pick, 3B-q4_K_M, and the 90% CI limits any 3B-q4_K_M deficit to 2.4 points. Yet 7B-q2_K needs 2.03 times the energy per solved task on CPU, GPU and memory rails, or 1.66 to 1.81 times on whole-system power. On HumanEval+, five alternating session pairs give a ratio of 1.925 against 1.956 in the main grid. At 8 GB, 7B-q6_K buys a significant gain over the 2 GB pick at a higher energy price. From 1.5B up, q4_K_M trails the highest precision measured at its size by less than 3 points (90% CI). We release per-task results, power logs and the analysis script.

## Making open models work for long-horizon agentic workloads

DevFeed: [Making open models work for long-horizon agentic workloads](<https://devfeed.tech/articles/making-open-models-work-for-long-horizon-agentic-workloads-141159.md>)

Original publisher: [Read original article](<https://ollama.com/blog/open-models-long-horizon-agents>)

Published: 2026-11-16T00:00:00Z

Content type: article

Language: en

Sources: [Ollama](<https://devfeed.tech/sources/ollama.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [coding assistant](<https://devfeed.tech/topics/coding-assistant.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [supervisor agent pattern](<https://devfeed.tech/topics/supervisor-agent-pattern.md>)

Tags: [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [agents](<https://devfeed.tech/tags/agents.md>), [api](<https://devfeed.tech/tags/api.md>), [cache](<https://devfeed.tech/tags/cache.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [conversations](<https://devfeed.tech/tags/conversations.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [prefix-caching](<https://devfeed.tech/tags/prefix-caching.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [routing](<https://devfeed.tech/tags/routing.md>)

### AI overview

Long coding-agent sessions repeatedly resend the full conversation while producing short tool calls, so preserving prefix-cache hits can reduce repeated prompt processing. The harness should keep early prompt content stable, place changing session details in the task message, choose reasoning effort at the start, disclose when tool output is truncated, and avoid giving subagents irrelevant history. Loading tools at the end can preserve the cached prefix, but some model templates do not support it. The inference engine must route turns to a node holding the cache and keep idle caches available; routing may choose a cache miss if the warm node is overloaded. The article describes offloading idle caches to CPU memory, paged cache allocation, continuous batching, and separate prefill and decode clusters. Caching reduces recomputation but is not free: cached tokens still consume memory and attention work.

### Source excerpt

Prefix caching helps coding agents stay fast as conversations grow. Prompts, tool calls, and inference routing determine how much work the model can reuse.

## Making open models work for long-horizon agentic workloads

DevFeed: [Making open models work for long-horizon agentic workloads](<https://devfeed.tech/articles/making-open-models-work-for-long-horizon-agentic-workloads-138313.md>)

Original publisher: [Read original article](<https://www.together.ai/blog/making-open-models-work-for-long-horizon-agentic-workloads>)

Author: Zain Hasan, Parth Sareen (Ollama)

Published: 2026-11-16T00:00:00Z

Content type: article

Language: en

Sources: [Together.ai](<https://devfeed.tech/sources/together-ai.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-apps](<https://devfeed.tech/tags/agentic-apps.md>), [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [agents](<https://devfeed.tech/tags/agents.md>), [api](<https://devfeed.tech/tags/api.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [calendar](<https://devfeed.tech/tags/calendar.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [codex](<https://devfeed.tech/tags/codex.md>), [coding](<https://devfeed.tech/tags/coding.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [harness](<https://devfeed.tech/tags/harness.md>), [models](<https://devfeed.tech/tags/models.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [open](<https://devfeed.tech/tags/open.md>), [session](<https://devfeed.tech/tags/session.md>), [together-ai](<https://devfeed.tech/tags/together-ai.md>)

### AI overview

Agent workloads resend a growing conversation with each turn, so prefix caching can reduce repeated prompt processing, but only when both the harness preserves the prompt prefix and the inference engine retains and routes to the cached session. The harness should keep system prompts and tool definitions stable, place changing session details in the task message, choose reasoning effort early, disclose when tool output is truncated, and avoid giving subagents irrelevant history. Late-loaded tool definitions can preserve the prefix when appended, though some chat templates do not support them there. On the engine side, routing may use a session key or the longest cached prefix; load can make a cache miss preferable to waiting on an overloaded node. Engines can offload idle caches to CPU memory, use paged allocation and continuous batching, and separate prefill from decode. Caching avoids recomputing old prompt work, but cached tokens still consume memory and are attended to during generation.

### Source excerpt

Every agent turn resends the full session. Together AI and Ollama break down how to protect the prefix cache, from the harness to the GPU.

## My Local AI Agent Needed a Bigger Brain, So I Gave It a Decentralized GPU

DevFeed: [My Local AI Agent Needed a Bigger Brain, So I Gave It a Decentralized GPU](<https://devfeed.tech/articles/my-local-ai-agent-needed-a-bigger-brain-so-i-gave-it-a-decentralized-gpu-134162.md>)

Original publisher: [Read original article](<https://hackernoon.com/my-local-ai-agent-needed-a-bigger-brain-so-i-gave-it-a-decentralized-gpu?source=rss>)

Author: zkzk gamal

Published: 2026-11-14T18:00:02Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [function calling](<https://devfeed.tech/topics/function-calling.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [ast](<https://devfeed.tech/tags/ast.md>), [backend](<https://devfeed.tech/tags/backend.md>), [calculator](<https://devfeed.tech/tags/calculator.md>), [decentralize-ai-hackathon](<https://devfeed.tech/tags/decentralize-ai-hackathon.md>), [decentralized](<https://devfeed.tech/tags/decentralized.md>), [decentralized-gpu](<https://devfeed.tech/tags/decentralized-gpu.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hackernoon-top-story](<https://devfeed.tech/tags/hackernoon-top-story.md>), [local](<https://devfeed.tech/tags/local.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [local-llm](<https://devfeed.tech/tags/local-llm.md>), [nosana](<https://devfeed.tech/tags/nosana.md>), [zkzkagent](<https://devfeed.tech/tags/zkzkagent.md>)

### AI overview

The article describes adding Nosana GPU inference to zkzkAgent while keeping routing, tools, files and confirmations on the user's machine. The agent can use local Ollama by default or send model calls to Nosana through LangChain's ChatOpenAI client; a live-model lookup selects a currently served model when none is configured. In a reported trial, a 27B Nosana model planned and built a FastAPI calculator app through 10 sequential local tool calls after plan approval; the app loaded and its API responded successfully. The author also reports safe expression parsing with Python's ast module and input limits. The arrangement lets sensitive work stay local while larger-model tasks use Nosana, but model calls do leave the machine in that mode. The article identifies remaining risks: shell commands can run without confirmation, and plan-level confirmations need strengthening. Benchmarks and automatic fallback are proposed future work, not reported results.

### Source excerpt

Running everything locally has a ceiling, and I kept hitting it. Here's how I fixed it.

## Running an AI Agent on a Local LLM: What I Measured on One GPU

DevFeed: [Running an AI Agent on a Local LLM: What I Measured on One GPU](<https://devfeed.tech/articles/running-an-ai-agent-on-a-local-llm-what-i-measured-on-one-gpu-133711.md>)

Original publisher: [Read original article](<https://www.sitepoint.com/running-an-ai-agent-on-a-local-llm-what-i-measured-on-one-gpu/>)

Author: Talha Anwar

Published: 2026-11-13T18:03:55Z

Content type: news

Language: en

Sources: [SitePoint](<https://devfeed.tech/sources/sitepoint.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents-md](<https://devfeed.tech/tags/agents-md.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-assistant](<https://devfeed.tech/tags/ai-assistant.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [automation](<https://devfeed.tech/tags/automation.md>), [bot](<https://devfeed.tech/tags/bot.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [chat](<https://devfeed.tech/tags/chat.md>), [compose](<https://devfeed.tech/tags/compose.md>), [config](<https://devfeed.tech/tags/config.md>), [container](<https://devfeed.tech/tags/container.md>), [context](<https://devfeed.tech/tags/context.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [discord](<https://devfeed.tech/tags/discord.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [local](<https://devfeed.tech/tags/local.md>)

### AI overview

On one RTX 3090, OpenClaw with Ollama's qwen3.5:9b handled short questions and simple file tasks, but its overhead and occasional tool-avoidance limited reliability. A one-line question produced an 18,415-token prompt because each request included the system prompt, tool definitions, and workspace files; reading it took about 10 seconds. The first reply took 70 seconds, including 59 seconds to load the model, while later replies took 1.9-5 seconds with prompt caching. The chat and embedding models used 11.4 GB of GPU memory together. OpenClaw also called the model on a 30-minute heartbeat, though the author did not measure its load impact. The 9B model sometimes guessed about files instead of checking them; explicitly asking it to use its read tool corrected one such error. The test supports local use for simple tasks, while tool-dependent answers need checking. These measurements describe one configuration, not all hardware or models.

### Source excerpt

null Continue reading Running an AI Agent on a Local LLM: What I Measured on One GPU on SitePoint.

## Get Started With Meta Muse in Python: Spark and Glimmer

DevFeed: [Get Started With Meta Muse in Python: Spark and Glimmer](<https://devfeed.tech/articles/get-started-with-meta-muse-in-python-spark-and-glimmer-133361.md>)

Original publisher: [Read original article](<https://realpython.com/meta-muse-python/>)

Author: Philipp Acsany

Published: 2026-11-13T14:00:00Z

Content type: tutorial

Language: en

Sources: [Real Python](<https://devfeed.tech/sources/real-python.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Python](<https://devfeed.tech/topics/python.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>)

Tags: [images](<https://devfeed.tech/tags/images.md>), [meta](<https://devfeed.tech/tags/meta.md>), [meta-muse](<https://devfeed.tech/tags/meta-muse.md>), [muse](<https://devfeed.tech/tags/muse.md>), [muse-spark](<https://devfeed.tech/tags/muse-spark.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [python](<https://devfeed.tech/tags/python.md>), [spark](<https://devfeed.tech/tags/spark.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [tool](<https://devfeed.tech/tags/tool.md>)

### AI overview

Muse Spark and Glimmer can be called from Python with the OpenAI client because Meta's Model API uses its protocol. The tutorial demonstrates Responses and Chat Completions requests, streaming, image input, function calls, and Pydantic structured output. Spark always reasons; in the tutorial's tests, setting effort to "minimal" cut latency by more than half and cost by about 60% while answering the checked question correctly, though it still used reasoning tokens. Default-effort streaming had a 12-34 second first-token delay across four runs, and tool_choice supports only "auto"; some other OpenAI parameters fail. Spark is available through Meta's endpoint or OpenRouter at the same token prices, but Meta's API has no hard spending cap. Glimmer's Apache 2.0 weights can be hosted locally or rented; in one matching hosted test it cost about one-eighth as much as Spark for the same question, but the comparison is limited. Local Glimmer needs substantial memory, and its tutorial script was not tested because the test machine had no GPU.

### Source excerpt

Get started with Meta Muse in Python. Make your first Muse Spark call, then add streaming, images, and tool calls, and try the open-weights Glimmer.

## How to Check Local LLM Output with Ollama, Gemma 4, and Nimble

DevFeed: [How to Check Local LLM Output with Ollama, Gemma 4, and Nimble](<https://devfeed.tech/articles/how-to-check-local-llm-output-with-ollama-gemma-4-and-nimble-131789.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-to-check-local-llm-output-with-ollama-gemma-4-and-nimble/>)

Author: Shuai Guo

Published: 2026-11-13T12:00:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Ollama](<https://devfeed.tech/topics/ollama.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [gemma-4](<https://devfeed.tech/tags/gemma-4.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [llm-applications](<https://devfeed.tech/tags/llm-applications.md>), [local](<https://devfeed.tech/tags/local.md>), [local-llm](<https://devfeed.tech/tags/local-llm.md>), [nimble](<https://devfeed.tech/tags/nimble.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [slm](<https://devfeed.tech/tags/slm.md>)

### AI overview

The article builds a local workflow in Ollama where Gemma 4 E2B drafts three attendee replies from team messages and Nimble, Bespoke Labs' 9B decision model, reviews them as ACCEPT, REVISE_LOCALLY, or HUMAN_REVIEW. Python keeps accepted answers, retries only answers Nimble marks locally correctable using the same context and requirement but one question at a time, and flags cases lacking enough information for a person. In the example, Nimble catches Gemma's mistaken claim that the attendee reminder was sent; the focused retry correctly says it was cancelled pending speaker-list sign-off. The setup requires Ollama 0.35.0 or later for Nimble's decision endpoint. Both models run via Ollama's local API, keeping the supplied context on the machine. The article cautions that results depend on models, prompts, and task, and recommends testing on task-specific examples.

### Source excerpt

Gemma 4 E2B drafts the replies, Bespoke Labs' Nimble 9B picks accept, revise, or human review, and only flagged questions get a focused retry. The post How to Check Local LLM Output with Ollama, Gemma 4, and Nimble appeared first on Towards Data Science.

## A Risk in the Clouds: NetClaw on GCP

DevFeed: [A Risk in the Clouds: NetClaw on GCP](<https://devfeed.tech/articles/a-risk-in-the-clouds-netclaw-on-gcp-131812.md>)

Original publisher: [Read original article](<https://automateyournetwork.com/netclaw/netclaw-on-gcp/>)

Author: John Capobianco

Published: 2026-11-12T12:00:00Z

Content type: article

Language: en

Sources: [Automate Your Network](<https://devfeed.tech/sources/automate-your-network.md>)

Topics: [experience report](<https://devfeed.tech/topics/experience-report.md>), [Cloud Networking](<https://devfeed.tech/topics/cloud-networking.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Pull Request](<https://devfeed.tech/topics/pull-request.md>)

Tags: [1-3](<https://devfeed.tech/tags/1-3.md>), [17](<https://devfeed.tech/tags/17.md>), [2](<https://devfeed.tech/tags/2.md>), [22](<https://devfeed.tech/tags/22.md>), [3](<https://devfeed.tech/tags/3.md>), [ai](<https://devfeed.tech/tags/ai.md>), [article](<https://devfeed.tech/tags/article.md>), [border](<https://devfeed.tech/tags/border.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [full](<https://devfeed.tech/tags/full.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [ha](<https://devfeed.tech/tags/ha.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [live](<https://devfeed.tech/tags/live.md>), [members](<https://devfeed.tech/tags/members.md>), [netclaw](<https://devfeed.tech/tags/netclaw.md>), [network-automation](<https://devfeed.tech/tags/network-automation.md>), [one](<https://devfeed.tech/tags/one.md>), [openshell](<https://devfeed.tech/tags/openshell.md>), [region](<https://devfeed.tech/tags/region.md>), [risk](<https://devfeed.tech/tags/risk.md>), [teardown](<https://devfeed.tech/tags/teardown.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

The article reports that NetClaw Borders and Members ran in GCP, on-premises, hybrid setups, and a live three-region Risk, but warns the system is not production-ready because Border URLs have no authentication: anyone with one can use the HUD and Chat and spend the model API key. In the tested design, Members dial out to Borders; multiple Borders share one execution authority, and loss of quorum blocks new protected work rather than authorizing duplicate execution. Multi-region tests exposed latency-sensitive admission windows, slow failover, and a distant two-region quorum that could not admit work. OpenShell with DefenseClaw added audited blocking and sandbox isolation, with measured overhead below half a percent of the 4,500 ms admission window, though one leadership timing issue remained unexplained. The tested workloads were read-only. The article recommends experimenting in a throwaway GCP project, setting budget alerts (which do not cap spending), keeping Border URLs private, and fully deleting resources afterward. Users, SSO, and RBAC are roadmap work, not current features.

### Source excerpt

From one Border to a three-region Risk: the full NetClaw on GCP story, R01-R09C, with cloud and hybrid Members, HA, OpenShell, DefenseClaw, live evidence and lessons from teardown.

## Bring Your Own Model is now generally available in Visual Studio

DevFeed: [Bring Your Own Model is now generally available in Visual Studio](<https://devfeed.tech/articles/bring-your-own-model-is-now-generally-available-in-visual-studio-129371.md>)

Original publisher: [Read original article](<https://devblogs.microsoft.com/visualstudio/bring-your-own-model-is-now-generally-available-in-visual-studio/>)

Author: Praveen Sethuraman

Published: 2026-11-11T17:00:48Z

Content type: release

Language: en

Sources: [Visual Studio](<https://devfeed.tech/sources/visual-studio-blog.md>)

Topics: [Visual Studio](<https://devfeed.tech/topics/visual-studio.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [Microsoft](<https://devfeed.tech/topics/microsoft.md>)

Tags: [agent-mode](<https://devfeed.tech/tags/agent-mode.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [air-gapped](<https://devfeed.tech/tags/air-gapped.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api](<https://devfeed.tech/tags/api.md>), [api-key](<https://devfeed.tech/tags/api-key.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [byom](<https://devfeed.tech/tags/byom.md>), [copilot](<https://devfeed.tech/tags/copilot.md>), [custom](<https://devfeed.tech/tags/custom.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [deployments](<https://devfeed.tech/tags/deployments.md>), [developer-community](<https://devfeed.tech/tags/developer-community.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lm-studio](<https://devfeed.tech/tags/lm-studio.md>), [machine](<https://devfeed.tech/tags/machine.md>), [models](<https://devfeed.tech/tags/models.md>), [visual-studio](<https://devfeed.tech/tags/visual-studio.md>)

### AI overview

Bring Your Own Model is generally available in Visual Studio 18.12, letting users connect external, organizational, or local models without GitHub sign-in. Users can keep multiple named configurations for one provider and add OpenAI-compatible custom endpoints; model listings appear when the endpoint supplies them. Custom models can be configured for Completions, Responses, or Anthropic-compatible Messages, with settings for context size, output limits, tool calling, and vision. LM Studio local models can connect through its OpenAI-compatible server. Provider connections and Agent Mode use Visual Studio's resolved proxy configuration, and administrators can enable or disable BYOM by policy. To start, open Chat's model picker, choose Manage models, add a provider, configure it and select a model. Access depends on a reachable endpoint and satisfied authentication requirements; organizational policy may disable the feature.

### Source excerpt

Whether your team hosts its own AI models, you have an API key from a model provider, or you want to experiment with a model running on your machine, Bring Your Own Model (BYOM) lets you use those models without leaving Visual Studio. BYOM is now generally available in Visual Studio 18.12. Since the August [...] The post Bring Your Own Model is now generally available in Visual Studio appeared first on Visual Studio Blog.

## Ollama makes it easy to run AI models locally for free

DevFeed: [Ollama makes it easy to run AI models locally for free](<https://devfeed.tech/articles/ollama-makes-it-easy-to-run-ai-models-locally-for-free-127281.md>)

Original publisher: [Read original article](<https://www.techspot.com/downloads/7772-ollama.html>)

Author: Ivan Franco

Published: 2026-11-10T19:12:00Z

Content type: article

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [Ollama](<https://devfeed.tech/topics/ollama.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [llama](<https://devfeed.tech/topics/llama.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [free software](<https://devfeed.tech/topics/free-software.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [coding](<https://devfeed.tech/tags/coding.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-r1](<https://devfeed.tech/tags/deepseek-r1.md>), [download](<https://devfeed.tech/tags/download.md>), [free](<https://devfeed.tech/tags/free.md>), [free-and-open-source](<https://devfeed.tech/tags/free-and-open-source.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [gemma-3](<https://devfeed.tech/tags/gemma-3.md>), [linux](<https://devfeed.tech/tags/linux.md>), [llama](<https://devfeed.tech/tags/llama.md>), [macos](<https://devfeed.tech/tags/macos.md>), [models](<https://devfeed.tech/tags/models.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [qwen](<https://devfeed.tech/tags/qwen.md>)

### AI overview

Ollama is a free, MIT-licensed, open-source platform for downloading, customizing, and running LLMs on macOS, Linux, and Windows using your own CPU or GPU. After downloading a model, you can run it offline; downloading models, accessing cloud-hosted models, and using online integrations require internet access. Ollama supports custom GGUF and Safetensors models, with prompts and parameters configurable through a Modelfile, and offers a CLI, REST API, Docker integration, and Python and JavaScript libraries. The stated minimums include a modern x86-64 CPU with AVX2 and 8 GB RAM; a GPU is optional but can speed inference. The source recommends VRAM ranging from 8 GB for 7B-13B models to 48 GB or more for 70B+ models.

### Source excerpt

Ollama lets you download and run popular AI models like DeepSeek, Qwen, Gemma, and Llama directly on your PC, without subscriptions or sending your data to the cloud. Use it for chatting, coding, and other AI tasks, with support for Windows, macOS, and Linux. Read Entire Article

## Benchmarking Scikit-Ollama vs. Groq-Backed Scikit-LLM: Speed, Cost, and Accuracy

DevFeed: [Benchmarking Scikit-Ollama vs. Groq-Backed Scikit-LLM: Speed, Cost, and Accuracy](<https://devfeed.tech/articles/benchmarking-scikit-ollama-vs-groq-backed-scikit-llm-speed-cost-and-accuracy-125008.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/benchmarking-scikit-ollama-vs-groq-backed-scikit-llm-speed-cost-and-accuracy/>)

Author: Iván Palomares Carrascosa

Published: 2026-11-09T11:00:06Z

Content type: tutorial

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [groq](<https://devfeed.tech/topics/groq.md>), [SciKits](<https://devfeed.tech/topics/scikit.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data-privacy](<https://devfeed.tech/tags/data-privacy.md>), [dedicated-inference](<https://devfeed.tech/tags/dedicated-inference.md>), [groq](<https://devfeed.tech/tags/groq.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [local](<https://devfeed.tech/tags/local.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [speed](<https://devfeed.tech/tags/speed.md>), [vs](<https://devfeed.tech/tags/vs.md>)

### AI overview

The article compares local zero-shot sentiment classification with Ollama and Scikit-Ollama against a Groq cloud endpoint routed through Scikit-LLM, using a shared helper to measure execution time and accuracy. The local Llama 3 run classified 4 of 5 examples correctly; that tiny dataset is not a reliable basis for broad accuracy comparisons. The excerpt reports $0 cost for both tested options at the time of writing, near-instantaneous cloud inference, and local latency tied to GPU VRAM and token generation speed. Cloud setup is described as simpler, while local inference keeps data on the machine. Reproducing the benchmark requires Ollama and a downloaded model locally, or a Groq API key for the cloud run.

### Source excerpt

In this article, you will learn how to build a side-by-side benchmark of local and cloud LLM classifiers using Scikit-Ollama and a Groq-backed Scikit-LLM pipeline....

## Build Your First Multi-Tool AI Agent with Ollama and Python

DevFeed: [Build Your First Multi-Tool AI Agent with Ollama and Python](<https://devfeed.tech/articles/build-your-first-multi-tool-ai-agent-with-ollama-and-python-121286.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/build-your-first-multi-tool-ai-agent-with-ollama-and-python/>)

Author: Thomas Reid

Published: 2026-11-06T13:30:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [circuit-breaker](<https://devfeed.tech/tags/circuit-breaker.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [database](<https://devfeed.tech/tags/database.md>), [database-tools](<https://devfeed.tech/tags/database-tools.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [llm-applications](<https://devfeed.tech/tags/llm-applications.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [python](<https://devfeed.tech/tags/python.md>), [tool](<https://devfeed.tech/tags/tool.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

The example turns a single database lookup into a multi-step Ollama agent: Python and SQLite expose three narrow, read-only tools, and the model chooses which to call based on earlier results. The loop passes the full message history back to Ollama because it has no memory between turns, records tool calls and results for inspection, and stops after eight model turns. Parameterized queries, row limits, and JSON results constrain the tools, though row limits can hide relevant records in larger datasets. The author also recommends repeated checks against known answers, since a successful run does not show reliability. The excerpt flags risks including oversized tool results, ambiguous questions, incorrect tool selection or interpretation, and broad exception handling that can mask application failures; it describes this as a demonstration rather than production-ready software.

### Source excerpt

AI Agents for Beginners, Part 2: The agent picks between three database tools, uses each result to choose its next call, and stops at an eight-turn circuit-breaker if it gets stuck. The post Build Your First Multi-Tool AI Agent with Ollama and Python appeared first on Towards Data Science.

## I Ran My AI Judge on a Decentralized GPU. The Entire Bill Was Four Cents.

DevFeed: [I Ran My AI Judge on a Decentralized GPU. The Entire Bill Was Four Cents.](<https://devfeed.tech/articles/i-ran-my-ai-judge-on-a-decentralized-gpu-the-entire-bill-was-four-cents-114857.md>)

Original publisher: [Read original article](<https://hackernoon.com/i-ran-my-ai-judge-on-a-decentralized-gpu-the-entire-bill-was-four-cents?source=rss>)

Author: Chintan D.

Published: 2026-11-03T13:56:38Z

Content type: news

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [observability](<https://devfeed.tech/topics/observability.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [account](<https://devfeed.tech/tags/account.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-compute](<https://devfeed.tech/tags/ai-compute.md>), [ai-decisions](<https://devfeed.tech/tags/ai-decisions.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [argument](<https://devfeed.tech/tags/argument.md>), [bill](<https://devfeed.tech/tags/bill.md>), [decentralize-ai-hackathon](<https://devfeed.tech/tags/decentralize-ai-hackathon.md>), [decentralized](<https://devfeed.tech/tags/decentralized.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-cloud](<https://devfeed.tech/tags/gpu-cloud.md>), [gpu-rental](<https://devfeed.tech/tags/gpu-rental.md>), [i](<https://devfeed.tech/tags/i.md>), [my](<https://devfeed.tech/tags/my.md>), [nosana](<https://devfeed.tech/tags/nosana.md>), [open-source-gpu-cloud](<https://devfeed.tech/tags/open-source-gpu-cloud.md>), [policy](<https://devfeed.tech/tags/policy.md>), [rtx](<https://devfeed.tech/tags/rtx.md>), [verifiable-ai](<https://devfeed.tech/tags/verifiable-ai.md>)

### AI overview

VeriTrace's evaluation model returned "incorrect" for a $5,000 refund that violated a $2,000 policy limit, and its verdict was signed with a separate judge key, bound to the receipt, and verified with the receipt signature and Merkle proof. The 7B Qwen2.5 model ran on a rented Nosana RTX 3090 over a public HTTPS endpoint; the judge wrapper itself still ran locally, so the full evaluation service was not decentralized. The settled charge was $0.04 for a 12.5-minute deployment at $0.19/hour, including idle time; an initially visible $1.05 was a reservation, not a charge. The first inference took about 74.7 seconds while loading model weights, and the next took 1.04 seconds. The run demonstrates a verifiable record and policy agreement in this case, not that the model's judgment is necessarily correct.

### Source excerpt

A 7B judge ran on a Nosana RTX 3090, caught a policy-breaking refund and signed its verdict into a receipt. The settled bill: four cents.

## Nace AI Open-Sources Drex Max v1: A 27B Decision Model That Runs Locally on Apple Silicon

DevFeed: [Nace AI Open-Sources Drex Max v1: A 27B Decision Model That Runs Locally on Apple Silicon](<https://devfeed.tech/articles/nace-ai-open-sources-drex-max-v1-a-27b-decision-model-that-runs-locally-on-apple-silicon-110625.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/01/nace-ai-open-sources-drex-max-v1-local-inference/>)

Author: Asif Razzaq

Published: 2026-11-01T08:10:07Z

Content type: article

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [MLX](<https://devfeed.tech/topics/mlx.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [all](<https://devfeed.tech/tags/all.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [api](<https://devfeed.tech/tags/api.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-m5](<https://devfeed.tech/tags/apple-m5.md>), [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [applications](<https://devfeed.tech/tags/applications.md>), [area](<https://devfeed.tech/tags/area.md>), [article](<https://devfeed.tech/tags/article.md>), [artificial-analysis](<https://devfeed.tech/tags/artificial-analysis.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [context-engineering](<https://devfeed.tech/tags/context-engineering.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [harness](<https://devfeed.tech/tags/harness.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [max](<https://devfeed.tech/tags/max.md>), [model](<https://devfeed.tech/tags/model.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [promote](<https://devfeed.tech/tags/promote.md>), [silicon](<https://devfeed.tech/tags/silicon.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [v1](<https://devfeed.tech/tags/v1.md>)

### AI overview

Nace AI has released open weights for Drex Max v1, a dense 27B decision model that returns probabilities for named choices, yes/no questions, or ordinal scores from text, JSON, or images. The excerpt describes uses such as routing, triage, and policy checks, with results available in one forward pass. It reports a self-reported 67.52 on Decision Index 0.4, versus 61.25 for Drex v2.0, and 73.92 on a public vision index. Local options include an affine Q4 build for Apple Silicon Macs with at least 24 GB and Q8_0 GGUF for 48 GB; CPU execution is also described. In Nace's tests, Q8_0 on a 48 GB M5 Pro stayed within 0.02 per probability of H100 bf16, while affine Q4 stayed within 0.0329 of Q8_0 with no choice flips. The excerpt says local Mac and CPU paths are text only for now; the hosted model supports images.

### Source excerpt

Nace AI open-sources Drex Max v1, a 27B multimodal decision model scoring 67.52 that runs on Apple Silicon. The post Nace AI Open-Sources Drex Max v1: A 27B Decision Model That Runs Locally on Apple Silicon appeared first on MarkTechPost.

## Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms

DevFeed: [Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms](<https://devfeed.tech/articles/introducing-anyspeechrecognizer-and-anydecisionmodel-open-models-for-speech-and-decisions-on-apple-platforms-107125.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/anydecisionmodel>)

Author: Mattt

Published: 2026-10-29T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [MLX](<https://devfeed.tech/topics/mlx.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Xcode](<https://devfeed.tech/topics/xcode.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>)

Tags: [1-5](<https://devfeed.tech/tags/1-5.md>), [22](<https://devfeed.tech/tags/22.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [alibaba](<https://devfeed.tech/tags/alibaba.md>), [all](<https://devfeed.tech/tags/all.md>), [allowed](<https://devfeed.tech/tags/allowed.md>), [also](<https://devfeed.tech/tags/also.md>), [alternatives](<https://devfeed.tech/tags/alternatives.md>), [ami](<https://devfeed.tech/tags/ami.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analyzer](<https://devfeed.tech/tags/analyzer.md>), [apple](<https://devfeed.tech/tags/apple.md>), [audio](<https://devfeed.tech/tags/audio.md>), [community](<https://devfeed.tech/tags/community.md>), [coreml](<https://devfeed.tech/tags/coreml.md>), [mlx](<https://devfeed.tech/tags/mlx.md>), [models](<https://devfeed.tech/tags/models.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [platforms](<https://devfeed.tech/tags/platforms.md>), [source](<https://devfeed.tech/tags/source.md>), [speech](<https://devfeed.tech/tags/speech.md>), [swift](<https://devfeed.tech/tags/swift.md>)

### AI overview

AnySpeechRecognizer gives Apple's SpeechAnalyzer-style Swift API a choice of speech engines, while AnyDecisionModel returns typed answers and probabilities without generating answer text. AnySpeechRecognizer supports Apple's engine, local Core ML Parakeet and Qwen3-ASR, and speaker diarization with Sortformer. Parakeet supports live input, volatile results, word timing, and confidence; Qwen3-ASR covers more languages and dialects but is slower and returns final segment results without those attributes. AnyDecisionModel offers next-token backends (MLX, Core AI, llama.cpp, Jev, and Ollama) and trained decision heads such as Clef. Next-token choices are limited to 26 options and scores to 10 levels; calibration and optional rotation debiasing address overconfidence and option-position bias, with added evaluation or computation costs. Both packages require Swift 6.2 and Apple OS 26 or later; local model and platform requirements vary. Measurements were made on one Mac and a few datasets, and the authors report testing only on macOS 27, not iPhone.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Ollama 0.40.11 Hands-On: Running Local LLMs on Windows

DevFeed: [Ollama 0.40.11 Hands-On: Running Local LLMs on Windows](<https://devfeed.tech/articles/ollama-0-40-11-hands-on-running-local-llms-on-windows-105446.md>)

Original publisher: [Read original article](<https://www.storagereview.com/review/ollama-0-40-11-hands-on-running-local-llms-on-windows>)

Author: Lyle Smith

Published: 2026-10-28T19:11:35Z

Content type: article

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [Ollama](<https://devfeed.tech/topics/ollama.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>)

Tags: [also](<https://devfeed.tech/tags/also.md>), [application](<https://devfeed.tech/tags/application.md>), [chat](<https://devfeed.tech/tags/chat.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-tools](<https://devfeed.tech/tags/coding-tools.md>), [command](<https://devfeed.tech/tags/command.md>), [command-line](<https://devfeed.tech/tags/command-line.md>), [command-line-interface](<https://devfeed.tech/tags/command-line-interface.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [end-user](<https://devfeed.tech/tags/end-user.md>), [files](<https://devfeed.tech/tags/files.md>), [first](<https://devfeed.tech/tags/first.md>), [gives](<https://devfeed.tech/tags/gives.md>), [image](<https://devfeed.tech/tags/image.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local](<https://devfeed.tech/tags/local.md>), [local-llms](<https://devfeed.tech/tags/local-llms.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [running](<https://devfeed.tech/tags/running.md>), [software](<https://devfeed.tech/tags/software.md>), [windows](<https://devfeed.tech/tags/windows.md>)

### AI overview

On a Windows 11 workstation with an RTX 5000 Ada, Ollama 0.40.11 ran two imported Jan GGUF models with automatic full GPU offload and no extra CUDA setup. Median decode speeds were 145.48 tokens/sec for Jan V3.5 and 92.87 for Jan V2 VL High, leading the tested series; comparisons reflect each app's runtime and offload behavior, and Ollama's requested batch size could not be verified. The desktop chat was easy to start, and its local API worked as soon as the service launched. Importing existing GGUFs and changing settings required PowerShell and Modelfiles, while the GUI showed less model detail and control than Jan. Image input worked with Ollama's downloaded Qwen3-VL 4B, but the imported Jan V2 model lacked its separate vision projector; screenshots also had to be saved before upload. Web Search required an account and failed with HTTP 402, whose cause the testers could not determine. The findings reflect one workstation and the versions tested.

### Source excerpt

Ollama 0.40.11 gives Windows users a desktop chat application and a local model runtime for other software. Ollama has also moved beyond its original command-line interface, adding a searchable model selector, image attachments, and links to coding tools. Some controls we needed for testing still live in PowerShell and model configuration files. We installed Ollama The post Ollama 0.40.11 Hands-On: Running Local LLMs on Windows appeared first on StorageReview.com.

## Spring AI 2.1 Message Parts: Reasoning That Survives the Round Trip

DevFeed: [Spring AI 2.1 Message Parts: Reasoning That Survives the Round Trip](<https://devfeed.tech/articles/spring-ai-2-1-message-parts-reasoning-that-survives-the-round-trip-104419.md>)

Original publisher: [Read original article](<https://spring.io/blog/2026/10/28/spring-ai-message-parts-reasoning-replay>)

Author: dimitarproynov

Published: 2026-10-28T00:00:00Z

Content type: article

Language: en

Sources: [Spring](<https://devfeed.tech/sources/spring.md>)

Topics: [Spring AI](<https://devfeed.tech/topics/spring-ai.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [function calling](<https://devfeed.tech/topics/function-calling.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Chat completions endpoints](<https://devfeed.tech/topics/chat-completions-endpoints.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api](<https://devfeed.tech/tags/api.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bedrock](<https://devfeed.tech/tags/bedrock.md>), [boot](<https://devfeed.tech/tags/boot.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [event-driven](<https://devfeed.tech/tags/event-driven.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [genai](<https://devfeed.tech/tags/genai.md>), [google](<https://devfeed.tech/tags/google.md>), [java](<https://devfeed.tech/tags/java.md>), [message](<https://devfeed.tech/tags/message.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [openai](<https://devfeed.tech/tags/openai.md>), [order](<https://devfeed.tech/tags/order.md>), [parts](<https://devfeed.tech/tags/parts.md>), [reactive](<https://devfeed.tech/tags/reactive.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reasoning-models](<https://devfeed.tech/tags/reasoning-models.md>), [responses](<https://devfeed.tech/tags/responses.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [spring](<https://devfeed.tech/tags/spring.md>), [spring-ai](<https://devfeed.tech/tags/spring-ai.md>), [text](<https://devfeed.tech/tags/text.md>), [thinking](<https://devfeed.tech/tags/thinking.md>), [thought](<https://devfeed.tech/tags/thought.md>), [web-applications](<https://devfeed.tech/tags/web-applications.md>)

### AI overview

Spring AI 2.1 replaces messages' text-plus-side-lists model with ordered message parts, preserving reasoning, tool calls, media and provider payloads across requests and streaming. The key rules are to keep parts in order, replay each provider's payload unchanged, and persist parts rather than text. This matters because providers impose different requirements: Anthropic and Gemini can reject missing or altered thinking signatures; OpenAI Responses requires prior reasoning items and tool exchanges for replay; DeepSeek and Mistral require reasoning in certain tool loops. Cross-provider histories retain text and tool calls but do not forward another provider's signed reasoning. Spring AI Session stores complete parts and tool-loop rounds as JSON, while the older JDBC chat-memory repository stores only text and drops tool calls and results. Earlier histories lack reasoning parts, and custom storage must retain part attributes for Mistral and unsigned Bedrock reasoning replay.

### Source excerpt

Spring AI: Message Parts Doc | Upgrade Notes | OpenAI Responses Doc | Session Doc Reasoning models do more than think before they answer. They hand part of that thinking back to you and expect to get it back on the next request. Claude signs its thinking blocks. Gemini attaches thought signatures to its function calls. OpenAI's Responses API returns encrypted reasoning items. If you drop, reorder or edit them, the result ranges from an HTTP 400 to a model that quietly loses its earlier reasoning and starts over. Until now, a Spring AI message was a piece of text with side lists of tool calls and media. There was no place for that data, let alone its order. Spring AI 2.1 changes the data model itself: every message now holds its content as an ordered list of message parts. A part is one piece of a response, such as a run of text, a block of reasoning or a tool call. Providers call these pieces content blocks (Anthropic, Bedrock), parts (Gemini), items (OpenAI Responses) or chunks (Mistral), and Spring AI maps them all to message parts. With the 2.1.0-RC1 release candidate, the major chat models produce and consume parts: OpenAI Responses, Anthropic, Google GenAI, OpenAI Chat Completions, Ollama, Bedrock Converse, DeepSeek and Mistral AI. The release also adds Session memory, which persists them. This post covers the new abstraction, why the Responses API needed it, and what each chat model gains. It comes down to three rules: keep the order, replay what you were given verbatim, and persist the parts, not the text. Why a String Was Not Enough Here is what Claude, with thinking enabled, returns in a single tool-calling round: thinking "The user wants the weather in Paris. I should call get_weather..." signature: "EqQBCgIYAhIM..." text "Let me check that for you." tool_use get_weather {"location": "Paris"} When you send the tool result, Anthropic wants this turn back exactly as it was: same blocks, same order, signature included. Before 2.1, AssistantMessage could not d

## Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models

DevFeed: [Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models](<https://devfeed.tech/articles/liquid-ai-releases-lfm3-6-open-weight-2b-class-multimodal-models-102602.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/10/27/liquid-ai-lfm3-open-weight-2b-multimodal-models/>)

Author: Asif Razzaq

Published: 2026-10-27T21:08:02Z

Content type: release

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [grpo](<https://devfeed.tech/topics/grpo.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-tasks](<https://devfeed.tech/tags/agentic-tasks.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [attention](<https://devfeed.tech/tags/attention.md>), [audio](<https://devfeed.tech/tags/audio.md>), [automationbench](<https://devfeed.tech/tags/automationbench.md>), [b300](<https://devfeed.tech/tags/b300.md>), [backend](<https://devfeed.tech/tags/backend.md>), [coding](<https://devfeed.tech/tags/coding.md>), [context](<https://devfeed.tech/tags/context.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [liquid-ai](<https://devfeed.tech/tags/liquid-ai.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [text](<https://devfeed.tech/tags/text.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Liquid AI's LFM3 release comprises six open-weight, roughly 2B-class models sharing a tri-hybrid backbone, with text, vision, audio, multimodal, and decision variants. The LFM3-1.7B text model has a 256K context window and is reported to use about 1 GB at 4-bit; the family includes GGUF, MLX, ONNX, FP8, and NVFP4 checkpoints. Liquid AI reports strong results on some agent benchmarks, including BFCL V4 and WebArena-Verified versus Qwen3.5-9B, while the smaller model trails it on GPQA Diamond, SWE-bench Verified, and 128K RULER. Other variants target video-capable vision, multilingual speech, combined vision/audio/language, and one-pass decisions. Deployment options include local runtimes and cloud services; the text notes that 128K context requires 1.6 GB of KV cache in the cited on-device setup.

### Source excerpt

Liquid AI released LFM3, 6 open-weight 2B-class models for text, vision, audio, omni and decisions. LFM3-1.7B uses a tri-hybrid backbone, a 256K context window and fits in about 1 GB at 4-bit. The post Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models appeared first on MarkTechPost.

## Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them

DevFeed: [Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them](<https://devfeed.tech/articles/broken-on-arrival-non-functional-quantized-llm-artifacts-in-public-model-registries-and-how-to-catch-them-101723.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.05881>)

Author: Aditi Patodiya

Published: 2026-10-27T04:00:00Z

Content type: news

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [artifacts](<https://devfeed.tech/tags/artifacts.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [files](<https://devfeed.tech/tags/files.md>), [functional](<https://devfeed.tech/tags/functional.md>), [functional-testing](<https://devfeed.tech/tags/functional-testing.md>), [gguf](<https://devfeed.tech/tags/gguf.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [library](<https://devfeed.tech/tags/library.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [original](<https://devfeed.tech/tags/original.md>), [public](<https://devfeed.tech/tags/public.md>), [registries](<https://devfeed.tech/tags/registries.md>)

### AI overview

The audit found seven of 305 eligible code-capable GGUF tags in the official Ollama library (2.3%) solved at most six of 164 HumanEval+ tasks; all were low-bit quantizations of models with at most four billion parameters. Five suspect files matched uncalibrated rebuilds tensor for tensor, showing that reproducibility did not guarantee functionality. For three Qwen2.5-Coder-3B builds, adding an importance matrix increased solved tasks from zero to 76-113 with the weights, toolchain, and tensor mix held fixed, while the same uncalibrated recipe left Qwen2.5-Coder-1.5B functional. None of 66 sampled tags that passed the initial 15-task screen proved non-functional. The authors release audit data and quantcheck, and recommend screening low-bit tags, then calibrating or withholding failures.

### Source excerpt

arXiv:2609.05881v2 Announce Type: replace Abstract: Developers download quantized large language models (LLMs) from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capable GGUF tags (the llama.cpp format) across 15 model lines in the official Ollama library, plus 22 popular Hugging Face community files. A 15-task code screen identified suspects, which we retested on all 164 HumanEval+ tasks. Seven official tags (2.3%) solved at most six of them. All were low-bit quantizations of models with at most four billion parameters. We rebuilt five suspect files from the original weights. All five registry files matched our uncalibrated rebuilds tensor for tensor, so none was corrupted. Exact reproducibility did not imply function. With the weights, toolchain and tensor mix held fixed, adding an importance matrix raised three Qwen2.5-Coder-3B builds from zero to 76-113 solved tasks. The same uncalibrated recipe left the smaller Qwen2.5-Coder-1.5B functional, showing that the recipe alone does not predict failure. None of 66 sampled tags that passed the screen was non-functional. We release the audit data and the quantcheck tool and recommend screening low-bit tags before publication, then calibrating or withholding failures.

## Lenovo ThinkPad P1 Gen 9 Review: Panther Lake and a 50W RTX PRO 2000 in the Thin ThinkPad Flagship

DevFeed: [Lenovo ThinkPad P1 Gen 9 Review: Panther Lake and a 50W RTX PRO 2000 in the Thin ThinkPad Flagship](<https://devfeed.tech/articles/lenovo-thinkpad-p1-gen-9-review-panther-lake-and-a-50w-rtx-pro-2000-in-the-thin-thinkpad-flagship-101894.md>)

Original publisher: [Read original article](<https://www.storagereview.com/review/lenovo-thinkpad-p1-gen-9-review-panther-lake-rtx-pro-2000>)

Author: Brian Beeler

Published: 2026-10-26T20:32:05Z

Content type: article

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [NVIDIA RTX](<https://devfeed.tech/topics/nvidia-rtx.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>)

Tags: [1-3](<https://devfeed.tech/tags/1-3.md>), [1-5](<https://devfeed.tech/tags/1-5.md>), [1080p](<https://devfeed.tech/tags/1080p.md>), [2-5gbe](<https://devfeed.tech/tags/2-5gbe.md>), [2026](<https://devfeed.tech/tags/2026.md>), [25h2](<https://devfeed.tech/tags/25h2.md>), [4k](<https://devfeed.tech/tags/4k.md>), [aluminum](<https://devfeed.tech/tags/aluminum.md>), [battery](<https://devfeed.tech/tags/battery.md>), [charging](<https://devfeed.tech/tags/charging.md>), [edge](<https://devfeed.tech/tags/edge.md>), [end-user](<https://devfeed.tech/tags/end-user.md>), [lenovo](<https://devfeed.tech/tags/lenovo.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [panther-lake](<https://devfeed.tech/tags/panther-lake.md>), [review](<https://devfeed.tech/tags/review.md>), [rtx](<https://devfeed.tech/tags/rtx.md>), [rtx-pro](<https://devfeed.tech/tags/rtx-pro.md>), [thinkpad](<https://devfeed.tech/tags/thinkpad.md>), [weight](<https://devfeed.tech/tags/weight.md>), [workstation](<https://devfeed.tech/tags/workstation.md>)

### AI overview

The ThinkPad P1 Gen 9 pairs a thin, sub-1.8kg workstation chassis with Panther Lake, a 50W RTX PRO 2000, Thunderbolt 5, and serviceable LPCAMM2 memory and dual M.2 storage. Against the prior-generation 285H system with the same GPU, it improves multi-threaded CPU results and memory bandwidth; its 50-minute? No: its 30-minute Cinebench loop retains 98.3% of its first-pass score, and its GPU leads the 47W RTX PRO 2000 by 19-27% on MLPerf token rates. Single-thread and browser results are largely unchanged. The 8GB GPU runs small local models quickly but cannot qualify for the review's 20B-and-larger model-load or coding-agent tiers, or serving tiers above 3B; higher-memory unified-memory systems handle those workloads. The review frames the P1 as suited to certified graphics and portable creative or engineering work, with local AI limited to small models. Its value depends on Lenovo's frequent discounts: the cited configuration was listed at $7,719 and sold for $5,399.

### Source excerpt

The Lenovo ThinkPad P1 Gen 9 is Lenovo's thin flagship mobile workstation. This 16-inch machine carries the premium features the rest of the ThinkPad P line trims to meet weight and price targets. It's an aluminum chassis that measures 9.87mm at the front edge and weighs under 1.8kg, a 90Wh battery with 140W charging, two The post Lenovo ThinkPad P1 Gen 9 Review: Panther Lake and a 50W RTX PRO 2000 in the Thin ThinkPad Flagship appeared first on StorageReview.com.

[Next page](<https://devfeed.tech/topics/ollama.md?cursor=WyIyMDI2LTEwLTI2VDIwOjMyOjA1KzAwOjAwIiwgIjNkZWRjYmVhLTVmNzMtNDAxNy05ZGI2LWZhYTZmYTMxNGY3NSJd>)