# model architecture Published articles for model architecture. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## IBM releases SOTA Granite Time Series PatchTST-FM-r2 model with commercial-friendly license DevFeed: [IBM releases SOTA Granite Time Series PatchTST-FM-r2 model with commercial-friendly license]() Original publisher: [Read original article]() Author: Roman Vaculin; Wesley M Gifford; Jiri Navratil; Chandra Reddy; Ayhan Sebin Published: 2026-09-09T15:36:24Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [foundation-models](), [Temporal data](), [releases](), [benchmarking](), [Open Source Models & Datasets](), [Inference]() Tags: [apache](), [benchmarking](), [foundation-models](), [ibm](), [inference](), [model-architecture](), [open-source](), [releases](), [time-series](), [zero-shot]() ### AI overview IBM released Granite Time Series PatchTST-FM-r2, a roughly 385M-parameter time-series foundation model for zero-shot forecasting. The article covers its architecture, probabilistic forecasting, missing-value imputation, benchmark results, licensing, and available reproducibility resources. ### Source excerpt Time-series foundation models are changing the way forecasting systems are built. Instead of training and maintaining a separate model for every dataset, users can use a pretrained model and generate forecasts zero-shot. IBM has released Granite Time Series PatchTST-FM-r2, the latest model in the Granite TSFM family (github, blog). ## 🗓 This Week In AI Research (1-7 August 26) DevFeed: [🗓 This Week In AI Research (1-7 August 26)]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-08-13T19:29:25Z Content type: article Language: en Sources: [Into AI]() Topics: [AI Research](), [releases](), [Benchmark](), [Large Language Model](), [Mixture of Experts (MoE)](), [model architecture](), [qwen]() Tags: [ai](), [ai-research](), [algorithm](), [benchmark](), [llms](), [mixture-of-experts](), [model-architecture](), [qwen](), [reasoning](), [releases]() ### AI overview A weekly roundup of AI research and model releases. It highlights Pathway, Bielik AI, and NYU's BDH-CQ reasoning model, which uses in-context learning with recurrent memory and latent-state reasoning, reports ARC-AGI-1 cost-efficiency results, and describes Alibaba's Qwen3.8-Max release and the U-OPSD self-distillation algorithm. ### Source excerpt The top 10 AI research papers and releases that you must know about this week. ## AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips DevFeed: [AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips]() Original publisher: [Read original article]() Author: Louise Ping; John Gray; Emily Webber; Josh Longenecker Published: 2026-08-10T20:23:04Z Content type: article Language: en Sources: [Amazon Science homepage]() Topics: [AI Chat]() Tags: [aws-trainium](), [chip-design](), [hardware](), [kernels](), [language-models](), [machine-learning](), [model-architecture](), [neurips](), [performance](), [training]() ### AI overview AWS Trainium Frontier is a competition for training language models from scratch on Trainium while co-designing architectures, optimizers, training loops, and optional custom kernels under fixed compute and time budgets. ### Source excerpt A competition with a finalist ceremony during NeurIPS 2026, challenging researchers to train language models from scratch on Trainium, exploring what optimal architectures look like when the hardware changes. ## Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference DevFeed: [Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-31T22:16:17Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Inference Performance](), [long-context](), [Artificial Intelligence](), [GPU](), [GPU optimization](), [Large Language Model](), [Nvidia]() Tags: [agentic-ai-generative-ai](), [ai](), [ai-inference](), [data-center-cloud](), [developer-tools-techniques](), [featured](), [gpu](), [inference](), [inference-performance](), [long-context](), [memory](), [model-architecture](), [nvidia](), [performance]() ### AI overview This article examines how co-designing dense attention with GPU execution can improve throughput and interactivity for long-context inference. It analyzes group size, head dimension, sequence length, and the different compute and memory behavior of prefill and decode, including the effects of speculative decoding and prefix caching. ### Source excerpt As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because... ## Making LLMs faster without sacrificing accuracy DevFeed: [Making LLMs faster without sacrificing accuracy]() Original publisher: [Read original article]() Author: Tao Yu; Youngsuk Park Published: 2026-05-15T13:00:00Z Content type: article Language: en Sources: [Amazon Science homepage]() Topics: [Language models](), [Training AI Models](), [web applications]() Tags: [architecture](), [chinchilla-scaling-law](), [conversational-ai](), [generative-ai](), [grouped-query-attention](), [hyperparameter-optimization](), [iclr-2026](), [inference](), [inference-efficiency](), [large-language-models](), [large-language-models-llms](), [llm-optimization](), [llms](), [machine-learning](), [model-architecture](), [network-architectures](), [scaling-laws](), [training](), [transformer-architecture]() ### AI overview The article presents scaling laws that connect LLM architectural choices to the tradeoff between accuracy and efficiency. It describes how these choices can improve inference throughput without reducing accuracy. ### Source excerpt A new scaling law that relates particular architectural choices to loss helps identify models that improve throughput by up to 47% with no loss of accuracy. ## Inside Praktika's conversational approach to language learning DevFeed: [Inside Praktika's conversational approach to language learning]() Original publisher: [Read original article]() Published: 2026-01-22T00:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [Artificial Intelligence](), [model architecture](), [App](), [real-time](), [data]() Tags: [agent](), [ai](), [app](), [architecture](), [building](), [data](), [education](), [gpt](), [model](), [model-architecture](), [product](), [real-time](), [startup]() ### AI overview Praktika uses GPT-powered personalized AI tutors and a multi-agent system to help learners build real-world language fluency through adaptive conversations, progress tracking, and long-term lesson planning. ### Source excerpt How Praktika uses GPT-4.1 and GPT-5.2 to build adaptive AI tutors that personalize lessons, track progress, and help learners achieve real-world language fluency ## Transformers v5: Simple model definitions powering the AI ecosystem DevFeed: [Transformers v5: Simple model definitions powering the AI ecosystem]() Original publisher: [Read original article]() Author: Lysandre; Arthur Zucker; Cyril Vallez; Vaibhav Srivastav Published: 2025-12-01T00:00:00Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [Transformers](), [hugging face](), [model architecture](), [Artificial Intelligence](), [Open Source]() Tags: [ai](), [announcement](), [community](), [hugging-face](), [inference](), [model-architecture](), [open-source](), [python](), [transformers](), [v5]() ### AI overview Hugging Face announces Transformers v5, focusing on simpler model definitions and improvements for training, inference, and production. The article describes the library's growth to more than 400 model architectures and over 1.2 billion installs, along with modular design intended to improve maintenance, integration, and collaboration. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## WeatherNext 2: Our most advanced weather forecasting model DevFeed: [WeatherNext 2: Our most advanced weather forecasting model]() Original publisher: [Read original article]() Author: The WeatherNext team Published: 2025-11-17T15:09:23Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [Artificial Intelligence](), [Google](), [model architecture](), [Inference](), [BigQuery](), [Google Cloud Platform (GCP)](), [AI Platform]() Tags: [ai](), [bigquery](), [google](), [inference](), [model](), [model-architecture](), [none](), [tpu](), [vertex-ai]() ### AI overview Google DeepMind and Google Research introduce WeatherNext 2, an AI weather forecasting model that generates hundreds of possible scenarios faster, at higher resolution, and with improved accuracy. Its forecast data is available through Earth Engine and BigQuery, with custom model inference offered through Vertex AI, and its technology is being integrated into several Google products. ### Source excerpt The new AI model delivers more efficient, more accurate and higher-resolution global weather predictions. ## NVIDIA Nemotron Nano v2: Long-Context Reasoning and Efficient Inference in Smaller Models DevFeed: [NVIDIA Nemotron Nano v2: Long-Context Reasoning and Efficient Inference in Smaller Models]() Original publisher: [Read original article]() Author: Alex Razvant Published: 2025-11-15T14:47:33Z Content type: article Language: en Sources: [Neural Bits]() Topics: [Nemotron](), [Artificial Intelligence](), [Nvidia](), [long-context](), [model architecture](), [Inference](), [Open Source](), [Transformer](), [Large Language Model]() Tags: [ai-ml](), [inference](), [long-context](), [model-architecture](), [moe](), [nemotron](), [nvidia](), [open-source](), [reasoning]() ### AI overview This technical article examines NVIDIA's Nemotron Nano v2 Transformer-Hybrid models, focusing on their long-context reasoning, fast inference, and reported performance against larger open models. It also describes the broader Nemotron family's open models, datasets, and fine-tuning recipes for agentic AI systems. ### Source excerpt How NVIDIA's Nemotron Nano V2 SLM, built for long-context reasoning, fast inference, can compete with models 4-5x its size. ## Fast LoRA inference for Flux with Diffusers and PEFT DevFeed: [Fast LoRA inference for Flux with Diffusers and PEFT]() Original publisher: [Read original article]() Author: Sayak Paul; Benjamin Bossan Published: 2025-07-23T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [flux](), [lora](), [Inference Performance](), [Hacktoberfest](), [diffusers](), [Optimization](), [peft](), [quantization](), [model architecture](), [text-to-image](), [GPU]() Tags: [amd](), [diffusers](), [diffusion](), [flux](), [gpu](), [guide](), [inference](), [latency](), [lora](), [model-architecture](), [optimization](), [peft](), [quantization](), [text-to-image]() ### AI overview This article presents an optimization recipe for faster LoRA inference with the Flux.1-Dev text-to-image model using Diffusers and PEFT. The approach addresses LoRA hotswapping and recompilation issues with Flash Attention 3, FP8 quantization from TorchAO, and hotswapping-ready compilation, achieving about a 2.3x speedup while balancing inference speed and memory use. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Mastering Long Contexts in LLMs with KVPress DevFeed: [Mastering Long Contexts in LLMs with KVPress]() Original publisher: [Read original article]() Author: Simon Jegou; Maximilian Jeblick Published: 2025-01-23T08:03:03Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Large Language Model](), [Compression](), [long-context](), [Cache](), [LLM Techniques](), [text-generation](), [model architecture]() Tags: [architecture](), [cache](), [compression](), [context-window](), [language-models](), [llms](), [long-context](), [model](), [model-architecture](), [models](), [text-generation](), [tokens]() ### AI overview This article introduces KVPress, an NVIDIA toolkit that applies KV cache compression techniques to make long-context Large Language Models (LLMs) more memory-efficient. It explains how context windows enable in-context retrieval, learning, and extended reasoning, and why KV Cache memory usage grows with context length. The article also describes how KV Cache reuses attention-layer keys and values during autoregressive text generation. ### Source excerpt TL;DR: KVPress packs the latest KV cache compression techniques, enabling memory-efficient long-context LLMs. 🚀 One of the key features of Large Language Models (LLMs) is their context window--the maximum number of tokens they can process in a single request. As LLMs evolve, their context windows are becoming increasingly larger. Larger context windows unlock incredible possibilities: - In-context retrieval: Seamlessly referencing large amounts of text within a single query. ## Client Time Series Model: a Multi-Target Recommender System based on Temporally-Masked Encoders DevFeed: [Client Time Series Model: a Multi-Target Recommender System based on Temporally-Masked Encoders]() Original publisher: [Read original article]() Published: 2022-10-14T06:00:00Z Content type: article Language: en Sources: [Stitch Fix]() Topics: [client](), [Time Series](), [recommendations](), [Embeddings](), [model architecture](), [maintenance](), [systems](), [data]() Tags: [data](), [embeddings](), [maintenance](), [model-architecture](), [recommendations](), [systems](), [time-series]() ### AI overview Stitch Fix describes its Client Time Series Model, a sequence-based recommender that estimates the probability of client-item purchases. The model uses a unified client embedding and incorporates the time dimension of client interactions to reduce duplicated models, improve maintainability, and share learning across business lines, regions, and channels. ### Source excerpt Introduction The foundation of our recommendation stack is a scoring model we call p(sale), which estimates the probability that any given client will purchase any given item. This model has gone through many iterations over the years, from a mixed effects model, to a matrix factorization model, and now to a novel sequence-based model. Internally we call this the Client Time Series Model (aka CTSM) because of its focus on the time-domain of client interactions. This post details our new model, which is a significant improvement for both the quality of our recommendations and the maintainability of our systems. Motivation Before setting out to develop our new model, it was clear that the evolution of our business necessitated a change to our modeling approach. First, the growing variety of recommendations we serve led to an explosion in the number of models we needed to maintain. Each time the business expanded, such as adding Mens, or serving the UK, or adding direct shopping with Freestyle, we responded by forking a new model to serve the new channel. This was necessary because a single domain-agnostic model could not serve the new channels as well as tailored models, but over time it has increased our maintenance burden and cost of iteration. In addition to the system complexity, we also knew we had an opportunity to make better use of important signals. With data and models separated by business line, region, and channel, we had a limited ability to leverage learning across these boundaries. With a unified model, we can more seamlessly use data from US clients to improve recommendations for UK clients, or data from Fixes to improve recommendations in Freestyle. Finally, our previous approaches modeled clients via tabular data. Although they are trained on purchase events that take place in the context of a particular point in time, they did not explicitly consider the time dimension in their understanding of the client's interactions. We believed that there was s ## Transformers for software engineers DevFeed: [Transformers for software engineers]() Original publisher: [Read original article]() Author: Nelson Elhage Published: 2022-04-01T20:00:00Z Content type: tutorial Language: en Sources: [Nelson Elhage]() Topics: [Transformer architecture](), [Programming](), [Software Engineering](), [Reverse Engineering](), [Transformers]() Tags: [anthropic](), [architecture](), [codex](), [deep-learning](), [gpt-3](), [model-architecture](), [programming](), [reverse-engineering](), [software-engineering](), [storm](), [transformer-architecture](), [transformers]() ### AI overview This tutorial explains Transformer architecture for software engineers, using software engineering and programming perspectives to discuss how GPT-style Transformer models work. It also connects the architecture to interpretability and reverse-engineering efforts. ### Source excerpt Ever since its introduction in the 2017 paper, Attention is All You Need, the Transformer model architecture has taken the deep-learning world by storm. Initially introduced for machine translation, it has become the tool of choice for a wide range of domains, including text, audio, video, and others. Transformers have also driven most of the massive increases in model scale and capability in the last few years. OpenAI's GPT-3 and Codex models are Transformers, as are DeepMind's Gopher models and many others. ## Deep NLP-based Recommenders at Finn.no DevFeed: [Deep NLP-based Recommenders at Finn.no]() Original publisher: [Read original article]() Author: Simen Eide Published: 2017-09-08T13:56:49Z Content type: tutorial Language: en Sources: [Finn.no]() Topics: [Natural language processing](), [Deep learning](), [Embeddings](), [recommendations](), [Keras](), [Hackathon]() Tags: [data](), [deep-learning](), [embeddings](), [hackathon](), [keras](), [model-architecture](), [neural-networks](), [nlp](), [recommendations]() ### AI overview The article describes a FINN.no hackathon project exploring deep NLP-based recommendations for classified ads. The team used categorized ad data, word embeddings, and a convolutional neural network architecture to model similarity, but the supplied text does not include the final performance results. ### Source excerpt During a hackathon at FINN.no, we figured we wanted to learn more about deep NLP-models. FINN.no has a large database with ads of people trying to sell stuff (around 1 million active ads at any time), and they are categorized into a category tree with three or four layers. For example, full suspension bikes can be found under "Sport and outdoor activities" / "Bike sport" / "Full suspension bikes". In our daily jobs we are working on recommendations. There, we already have a content based (tf-idf) recommender build on Solr's More Like This. It seems to work well in areas where our collaborative filtering approaches does not. Would it be possible to build a deep learning NLP-model of similar performance? To achieve a measure of similarity, building a classifier of the previously mentioned categories seemed like a good choice, since we already had a lot of pre-existing data. The NLP team at Schibsted had already tokenized around six million ads as well as trained a word2vec model for us - we were ready to roll! Some preprocessing still had to be done. We ran through all ads, concatenated the title and description strings, and after a quick look at the data took the first 15 words of each ad. Model architecture proposed by the paper Our initial experiments were done with a simple "Bag of words" model included in the Keras repository, but we promptly switched over to "Convolutional Neural Networks for Sentence Classification" based architecture after hearing about it from our colleague, Tobias. By looking at the first 15 words of the ad, and using 200 dimensional embeddings for each word, our input is transformed into a 15x200 matrix. We apply three different convolutions on each document. The three convolutions looks at 2, 3 and 4 words (kernel sizes) in each convolution. It then max-pools each over the whole document, so that you end up with one value per document per convolution. For each kernel size you do 100 different filters. Finally you add a dense layer for clas