# llama3

Published articles for llama3.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## A Glance at GPU Goodness in Java: LLM Inference with TornadoVM

DevFeed: [A Glance at GPU Goodness in Java: LLM Inference with TornadoVM](<https://devfeed.tech/articles/a-glance-at-gpu-goodness-in-java-llm-inference-with-tornadovm-23010.md>)

Original publisher: [Read original article](<https://www.javaadvent.com/2025/12/a-glance-at-gpu-goodness-in-java-llm-inference-with-tornadovm.html>)

Author: Edoardo Vacchi

Published: 2025-12-11T03:03:21Z

Content type: tutorial

Language: en

Sources: [Java Advent Calendar](<https://devfeed.tech/sources/java-advent-calendar.md>)

Topics: [TornadoVM](<https://devfeed.tech/topics/tornadovm.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Java](<https://devfeed.tech/topics/java.md>), [openjdk](<https://devfeed.tech/topics/openjdk.md>), [Programming](<https://devfeed.tech/topics/programming.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [c-c-plus-plus](<https://devfeed.tech/tags/c-c-plus-plus.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [java](<https://devfeed.tech/tags/java.md>), [jdk](<https://devfeed.tech/tags/jdk.md>), [linux](<https://devfeed.tech/tags/linux.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llm](<https://devfeed.tech/tags/llm.md>), [openjdk](<https://devfeed.tech/tags/openjdk.md>), [tornadovm](<https://devfeed.tech/tags/tornadovm.md>)

### AI overview

This tutorial introduces TornadoVM, an OpenJDK plugin that compiles annotated Java code for heterogeneous hardware, and explains how it can run LLM inference on GPUs. It covers installation, a basic GPU kernel, and the GPULlama3 project, which demonstrates running a Llama 3 model in Java with GPU acceleration.

### Source excerpt

It seems like it's become a tradition that I announce I have joined a new company for the Java Advent of Code. At least this time it's actually an old friend: I am excited to be back at Red Hat, in the llm-d team! Does that mean I forgot about Java? Of course not. If [...] The post A Glance at GPU Goodness in Java: LLM Inference with TornadoVM appeared first on JVM Advent.

## Red-teaming a RAG app: gpt-4o-mini v. llama3.1 v. hermes3

DevFeed: [Red-teaming a RAG app: gpt-4o-mini v. llama3.1 v. hermes3](<https://devfeed.tech/articles/red-teaming-a-rag-app-gpt-4o-mini-v-llama3-1-v-hermes3-21741.md>)

Original publisher: [Read original article](<http://blog.pamelafox.org/2025/08/red-teaming-rag-app-what-happens.html>)

Author: Pamela Fox (noreply@blogger.com)

Published: 2025-08-04T17:08:00Z

Content type: article

Language: en

Sources: [Pamela Fox](<https://devfeed.tech/sources/pamela-fox.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [Python](<https://devfeed.tech/topics/python.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [azure](<https://devfeed.tech/tags/azure.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openai](<https://devfeed.tech/tags/openai.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [python](<https://devfeed.tech/tags/python.md>), [rag](<https://devfeed.tech/tags/rag.md>), [red-teaming](<https://devfeed.tech/tags/red-teaming.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

This article examines red-teaming a retrieval-augmented generation application using an automated Red Teaming agent from the azure-ai-evaluations Python package. It describes how adversarial questions are generated, transformed with the open-source pyrit package, sent to a RAG-on-PostgreSQL application, and evaluated for unsafe responses across models including gpt-4o-mini, llama3.1, and hermes3.

### Source excerpt

When we develop user-facing applications that are powered by LLMs, we're taking on a big risk that the LLM may produce output that is unsafe in some way - like responses that encourage violence, hate speech, or self-harm. How can we be confident that a troll won't get our app to say something horrid? We could throw a few questions at it while manually testing, like "how do I make a bomb?", but that's only scratching the surface. Malicious users have gone to far greater lengths to manipulate LLMs into responding in ways that we definitely don't want happening in domain-specific user applications. Red-teaming That's where red teaming comes in: bring in a team of people that are expert at coming up with malicious queries and that are deeply familiar with past attacks, give them access to your application, and wait for their report of whether your app successfully resisted the queries. But red-teaming is expensive, requiring both time and people. Most companies don't have the resources nor expertise to have a team of humans red-teaming every app, plus every iteration of an app each time a model or prompt changes. Fortunately, my colleagues at Microsoft developed an automated Red Teaming agent, part of the azure-ai-evaluations Python package. The agent uses an adversarial LLM, housed safely inside an Azure AI Foundry project such that it can't be used for other purposes, in order to generate unsafe questions across various categories. The agent then transforms the questions using the open-source pyrit package, which uses known attacks like base-64 encoding, URL encoding, Ceaser Cipher, and many more. It sends both the original plain text questions and transformed questions to your app, and then evaluates the response to make sure that the app didn't actually answer the unsafe question. RAG application So I red-team'ed a RAG app! My RAG-on-PostgreSQL sample application answers questions about products from a database representing a fictional outdoors store. It uses a basi

## Run Large Language Models Locally on a Mac with Ollama

DevFeed: [Run Large Language Models Locally on a Mac with Ollama](<https://devfeed.tech/articles/ollama-29002.md>)

Original publisher: [Read original article](<https://nshipster.com/ollama/>)

Author: Mattt (mattt@nshipster.com)

Published: 2025-02-14T08:00:00Z

Content type: tutorial

Language: en

Sources: [NSHipster](<https://devfeed.tech/sources/nshipster.md>)

Topics: [Ollama](<https://devfeed.tech/topics/ollama.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [llama3](<https://devfeed.tech/topics/llama3.md>), [Containers](<https://devfeed.tech/topics/containers.md>), [Homebrew](<https://devfeed.tech/topics/homebrew.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [containers](<https://devfeed.tech/tags/containers.md>), [homebrew](<https://devfeed.tech/tags/homebrew.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llms](<https://devfeed.tech/tags/llms.md>), [miscellaneous](<https://devfeed.tech/tags/miscellaneous.md>), [oci](<https://devfeed.tech/tags/oci.md>), [ollama](<https://devfeed.tech/tags/ollama.md>)

### AI overview

This tutorial explains how to use Ollama to download, run, and manage large language models locally on a Mac. It covers installation, llama3.2, llama.cpp, Modelfiles, OCI-based model distribution, and the practical benefits of local inference, including privacy, cost, latency, control, and reliability.

### Source excerpt

While we wait for Apple Intelligence to arrive on our devices, something remarkable is already running on our Macs. Think of it as a locavore approach to artificial intelligence: homegrown, sustainable, and available year-round.

## Benchmarking Language Model Performance on 5th Gen Xeon at GCP

DevFeed: [Benchmarking Language Model Performance on 5th Gen Xeon at GCP](<https://devfeed.tech/articles/benchmarking-language-model-performance-on-5th-gen-xeon-at-gcp-7290.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/intel-gcp-c4>)

Author: Matrix Yao; Ke Ding; Ilyas Moutawwakil

Published: 2024-12-17T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [cpu](<https://devfeed.tech/topics/cpu.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [intel](<https://devfeed.tech/topics/intel.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [llama3](<https://devfeed.tech/topics/llama3.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [backend](<https://devfeed.tech/tags/backend.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [function-calling](<https://devfeed.tech/tags/function-calling.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google](<https://devfeed.tech/tags/google.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intel](<https://devfeed.tech/tags/intel.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llm](<https://devfeed.tech/tags/llm.md>), [performance](<https://devfeed.tech/tags/performance.md>), [text-generation](<https://devfeed.tech/tags/text-generation.md>)

### AI overview

This article benchmarks language-model performance on Google Cloud Compute Engine C4 and N2 instances powered by different generations of Intel Xeon processors. It compares text embedding and text generation for agentic AI workloads, focusing on the benefits of Intel Advanced Matrix Extensions (AMX) and CPU-based hosting of systems using small language models such as Meta's 1B and 3B Llama 3.2 models.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## "Llama 3.2 in Keras"

DevFeed: ["Llama 3.2 in Keras"](<https://devfeed.tech/articles/llama-3-2-in-keras-7302.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/keras-llama-32>)

Author: Martin Görner

Published: 2024-10-21T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Keras](<https://devfeed.tech/topics/keras.md>), [llama3](<https://devfeed.tech/topics/llama3.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [tokenization](<https://devfeed.tech/topics/tokenization.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Tensorflow](<https://devfeed.tech/topics/tensorflow.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [keras](<https://devfeed.tech/tags/keras.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [tensorflow](<https://devfeed.tech/tags/tensorflow.md>), [tokenization](<https://devfeed.tech/tags/tokenization.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The article introduces Llama 3.2 support in Keras, explaining that Hugging Face checkpoints can be loaded directly and converted on the fly when needed. It also describes Keras and keras-hub support for multiple backends, pretrained models, tokenization, preprocessing, training, and fine-tuning.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Fine-tuning LLMs to 1.58bit: extreme quantization made easy

DevFeed: [Fine-tuning LLMs to 1.58bit: extreme quantization made easy](<https://devfeed.tech/articles/fine-tuning-llms-to-1-58bit-extreme-quantization-made-easy-6973.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/1_58_llm_extreme_quantization>)

Author: Mohamed Mekkouri; Marc Sun; Leandro von Werra; Pedro Cuenca; Omar Sanseviero; Thomas Wolf

Published: 2024-09-18T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [community](<https://devfeed.tech/tags/community.md>), [energy](<https://devfeed.tech/tags/energy.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llms](<https://devfeed.tech/tags/llms.md>), [model](<https://devfeed.tech/tags/model.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [performance](<https://devfeed.tech/tags/performance.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [research](<https://devfeed.tech/tags/research.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The article describes fine-tuning Llama models with the BitNet architecture at 1.58-bit ternary precision. It covers the resulting efficiency claims, benchmark comparison, and Transformers integration through BitLinear layers and a new "bitnet" quantization method.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing The AWS AI Stack

DevFeed: [Introducing The AWS AI Stack](<https://devfeed.tech/articles/introducing-the-aws-ai-stack-14082.md>)

Original publisher: [Read original article](<https://www.serverless.com/blog/aws-ai-stack>)

Author: Serverless Team

Published: 2024-09-11T00:00:00Z

Content type: release

Language: en

Sources: [Serverless Blog](<https://devfeed.tech/sources/serverless-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [AWS Lambda](<https://devfeed.tech/topics/aws-lambda.md>), [Boilerplate](<https://devfeed.tech/topics/boilerplate.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [React](<https://devfeed.tech/topics/react.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [Front end](<https://devfeed.tech/topics/frontend.md>), [llama3](<https://devfeed.tech/topics/llama3.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-lambda](<https://devfeed.tech/tags/aws-lambda.md>), [back-end](<https://devfeed.tech/tags/back-end.md>), [bedrock](<https://devfeed.tech/tags/bedrock.md>), [ci-cd](<https://devfeed.tech/tags/ci-cd.md>), [claude](<https://devfeed.tech/tags/claude.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [faas](<https://devfeed.tech/tags/faas.md>), [front-end](<https://devfeed.tech/tags/front-end.md>), [function-as-a-service](<https://devfeed.tech/tags/function-as-a-service.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llms](<https://devfeed.tech/tags/llms.md>), [news](<https://devfeed.tech/tags/news.md>), [react](<https://devfeed.tech/tags/react.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [serverless-architecture](<https://devfeed.tech/tags/serverless-architecture.md>), [serverless-framework](<https://devfeed.tech/tags/serverless-framework.md>)

### AI overview

The AWS AI Stack is introduced as a full-stack, serverless boilerplate for building AI applications on AWS. It uses Bedrock LLMs including Claude 3.5 Sonnet and Llama3.1, with a React front end, AWS Lambda back end, and built-in CI/CD.

### Source excerpt

Full-stack, serverless, boilerplate for AI applications on AWS, featuring Bedrock LLMs like Claude 3.5 Sonnet and Llama3.1, a React front-end, AWS Lambda back-end, built-in CI/CD and more.

## Deploy models on AWS Inferentia2 from Hugging Face

DevFeed: [Deploy models on AWS Inferentia2 from Hugging Face](<https://devfeed.tech/articles/deploy-models-on-aws-inferentia2-from-hugging-face-7285.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/inferentia-inference-endpoints>)

Author: Jeff Boudier; Philipp Schmid

Published: 2024-05-22T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [MLOps](<https://devfeed.tech/topics/mlops.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [model](<https://devfeed.tech/tags/model.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

Hugging Face announces support for deploying models on AWS Inferentia2 through Amazon SageMaker and Hugging Face Inference Endpoints. The update enables scalable inference for supported models, including Meta Llama 3, with multiple instance sizes, managed features, and autoscaling.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## License to Call: Introducing Transformers Agents 2.0

DevFeed: [License to Call: Introducing Transformers Agents 2.0](<https://devfeed.tech/articles/license-to-call-introducing-transformers-agents-2-0-7072.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/agents>)

Author: Aymeric Roucher; Lysandre; Pedro Cuenca

Published: 2024-05-13T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Bots](<https://devfeed.tech/topics/ai-bots.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [langchain](<https://devfeed.tech/tags/langchain.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mixtral](<https://devfeed.tech/tags/mixtral.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [smolagents](<https://devfeed.tech/tags/smolagents.md>), [tools](<https://devfeed.tech/tags/tools.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

Hugging Face announces Transformers Agents 2.0, an LLM-driven agent framework focused on iterative agents, tool use, modularity, clarity, and sharing. The article also notes its transition to the standalone smolagents library.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.