# pip pip is the package installer for Python, used to install packages from the Python Package Index and other indexes. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## PDFx - Extract references and metadata from PDF documents, and download all referenced PDFs DevFeed: [PDFx - Extract references and metadata from PDF documents, and download all referenced PDFs]() Original publisher: [Read original article]() Author: Chris Hager Published: 2026-09-17T02:48:44.791916Z Content type: tutorial Language: en Sources: [Chris Hager]() Topics: [pdf](), [Python](), [Command-line interface](), [JSON](), [pip](), [Open Source](), [GitHub]() Tags: [apache](), [broken-links](), [command-line](), [download](), [github](), [install](), [json](), [library](), [open-source](), [pdf](), [python]() ### AI overview PDFx is an Apache-licensed open-source tool that extracts references and metadata from PDF documents. It detects PDF, URL, arXiv, and DOI references, can download referenced PDFs in parallel, identify broken hyperlinks, extract PDF text, and output results as text or JSON. It is available as a command-line tool and Python package, with support for local and online PDFs. ### Source excerpt Reading over this paper and its references recently, I thought it would be great to be able to download all the references at once... This inspired me to write a little tool to do just that, and now it's done and released under the Apache open source license: https://github.com/metachris/pdfx Features Extract references and metadata from a given PDF Detects pdf, url, arxiv and doi references Fast, parallel download of all referenced PDFs Find broken hyperlinks (using the -c flag) (more) Output as text or JSON (using the -j flag) Extract the PDF text (using the --text flag) Use as command-line tool or Python package Compatible with Python 2 and 3 Works with local and online pdfs Getting Started Grab a copy of pdfx with easy_install or pip and run it: ## Quiz: An Effective Python Environment: Making Yourself at Home DevFeed: [Quiz: An Effective Python Environment: Making Yourself at Home]() Original publisher: [Read original article]() Author: Real Python Published: 2026-09-10T12:00:00Z Content type: article Language: en Sources: [Real Python]() Topics: [Python](), [Terminal](), [Command-line interface](), [pip](), [Poetry](), [coding]() Tags: [coding](), [dependencies](), [developers](), [poetry](), [python](), [terminal]() ### AI overview An interactive 13-question quiz tests understanding of effective Python development environments, covering shells and terminals, managing multiple Python versions, virtual environments, and dependency management with pip and Poetry. ### Source excerpt Test your understanding of Python environments, including shells, terminals, pyenv, virtual environments, and package managers like pip and poetry. ## Run High-Performance Core Math at Scale with NVIDIA nvmath-python DevFeed: [Run High-Performance Core Math at Scale with NVIDIA nvmath-python]() Original publisher: [Read original article]() Author: Michelle Horton Published: 2026-07-30T22:43:04Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Python](), [CUDA](), [math](), [GPU](), [NumPy](), [cpu](), [PyTorch](), [pip]() Tags: [agentic-ai-generative-ai](), [c-plus-plus](), [ci-cd](), [cpu](), [cuda](), [cuda-x](), [data-science](), [developer-tools-techniques](), [device-apis](), [featured](), [gpu](), [multi-gpu](), [nvidia](), [performance](), [python](), [pytorch](), [scale]() ### AI overview NVIDIA nvmath-python 1.0 provides a Pythonic interface to CUDA-X and NVPL math libraries, enabling optimized numerical operations on CPUs, CUDA GPUs, and distributed multi-GPU, multi-node systems. The article covers its sparse-tensor approach, flexible installation options, and interoperability with NumPy, CuPy, and PyTorch. ### Source excerpt NVIDIA nvmath-python is a library designed to bridge the gap between the Python scientific community and NVIDIA CUDA-X math libraries. It gives Python users... ## Fix the 'externally-managed-environment' Error on Raspberry Pi DevFeed: [Fix the 'externally-managed-environment' Error on Raspberry Pi]() Original publisher: [Read original article]() Author: Robert Lopez Published: 2026-07-01T05:00:00Z Content type: tutorial Language: en Sources: [RaspberryTips]() Topics: [pip](), [Python](), [apt](), [Debian](), [Cheat sheet]() Tags: [apt](), [debian](), [guide](), [guides](), [how-to](), [programming](), [python](), [raspberry-pi](), [tutorial]() ### AI overview This guide explains why pip installation can fail with an "externally-managed-environment" error on Raspberry Pi systems running Debian 12 or newer. It describes the role of APT in managing system Python packages and helps readers choose an appropriate installation method without damaging the system setup. ### Source excerpt Many older Raspberry Pi tutorials still show a simple pip install command to add Python packages. It worked for years, but on recent operating systems, it can now fail with an "externally-managed-environment" error. If this is the first time you've seen it, here's what changed and how to fix it. The externally-managed-environment error appears when... ## Introducing automatic, short-lived credentials for Chainguard Libraries for Python DevFeed: [Introducing automatic, short-lived credentials for Chainguard Libraries for Python]() Original publisher: [Read original article]() Published: 2026-01-15T00:00:00Z Content type: article Language: en Sources: [Chainguard: Unchained]() Topics: [chainguard libraries for python](), [chainguard libraries](), [Authentication](), [pip](), [developer velocity](), [Usability]() Tags: [authentication](), [chainguard-libraries](), [chainguard-libraries-for-python](), [chainguard-libraries-python](), [chainguard-packages](), [chainguard-python-containers](), [developer-velocity](), [github-actions](), [pypi](), [python-libraries](), [python-packages](), [secure-python-packages](), [security](), [usability]() ### AI overview Chainguard announces integrated authentication for Chainguard Libraries for Python through a keyring package. The package provides short-lived credentials that refresh automatically, enabling secure pip access without repeatedly creating, copying, or managing long-lived tokens. It supports local development and GitHub Actions environments with assumable identities. ### Source excerpt New integrated authentication for Python Libraries with a keyring: use short-lived credentials for pip installs to stay secure without slowing developers down. ## Santa's Python Pitfalls: A Java Developer's Guide to Staying Safe This Christmas DevFeed: [Santa's Python Pitfalls: A Java Developer's Guide to Staying Safe This Christmas]() Original publisher: [Read original article]() Author: Steve Poole Published: 2025-12-24T03:03:09Z Content type: tutorial Language: en Sources: [Java Advent Calendar]() Topics: [Python](), [pip](), [PyPI](), [Maven Central](), [Java](), [Maven]() Tags: [2025](), [attacks](), [dependency](), [developer](), [guide](), [install](), [internet](), [java](), [maven-central](), [mirror](), [model](), [packaging](), [pitfalls](), [pypi](), [python](), [registry](), [trust]() ### AI overview A Java-focused guide to Python package-management risks. It contrasts Maven Central with PyPI, highlighting flat package naming, trusting dependency resolution, typosquatting, dependency confusion, and unsafe installation commands. It recommends version pins, known sources, private registries, and lock files. ### Source excerpt Just like that it happened. You, a disciplined Java developer, are now installing Python. Like everything in 2025, it just arrived. One day, you were running a tidy mvn install, the next, you're learning about virtual environments and fighting an unfriendly pip install that won't explain what it just pulled from the internet. Good news: [...] The post Santa's Python Pitfalls: A Java Developer's Guide to Staying Safe This Christmas appeared first on JVM Advent. ## Sandbox Your AI Dev Tools: A Practical Guide for VMs and Lima DevFeed: [Sandbox Your AI Dev Tools: A Practical Guide for VMs and Lima]() Original publisher: [Read original article]() Author: Chris Hager Published: 2025-11-25T00:00:00Z Content type: tutorial Language: en Sources: [Chris Hager]() Topics: [AI-assisted coding](), [developer tooling](), [virtual machines](), [sensitive data](), [API keys](), [ssh](), [npm](), [pip]() Tags: [ai-assisted-coding](), [api-keys](), [dev-tools](), [npm](), [python](), [sandbox](), [sensitive-data](), [ssh]() ### AI overview A practical guide to sandboxing AI coding assistants and other development tools in isolated virtual machines using Lima. It explains how isolation helps protect SSH keys, API tokens, credentials, environment variables, and other sensitive data from arbitrary code, scripts, and package installation behavior. ### Source excerpt AI coding assistants, npm, pip, and other development tools can run arbitrary code and scripts on your machine, potentially stealing SSH keys, API tokens, wallet keys, sensitive credentials and other private data without you noticing. This guide shows you how to sandbox these tools in isolated VMs using Lima, so you can experiment and develop freely without putting your sensitive data at risk. Jump straight to the guide, or read on for a bit of personal context. ## Local Speed, Smooth Deploys: Heroku Adds Support for uv DevFeed: [Local Speed, Smooth Deploys: Heroku Adds Support for uv]() Original publisher: [Read original article]() Author: Ken W. Alger Published: 2025-06-04T15:00:06Z Content type: release Language: en Sources: [Heroku]() Topics: [Heroku](), [Package manager](), [Python](), [pip](), [Deployment]() Tags: [cloud-infrastructure](), [deployment](), [developer-tools](), [efficiency](), [engineering](), [heroku](), [package-management](), [packages](), [python]() ### AI overview Heroku adds support for uv, a Python package manager positioned as a faster alternative to pip. The article discusses Python package-management history and describes setting up a project for Heroku deployment with uv. ### Source excerpt Ah, another day, another deep dive into the ever-evolving world of Python development! Today, let's talk about something near and dear to every Pythonista's heart - managing those crucial external packages. For years, pip has been our trusty companion, the workhorse that gets the job done. But the landscape is shifting, and a new contender [...] The post Local Speed, Smooth Deploys: Heroku Adds Support for uv appeared first on Heroku. ## The ultimate guide to creating a secure Python package DevFeed: [The ultimate guide to creating a secure Python package]() Original publisher: [Read original article]() Author: Gourav Singh Bais Published: 2024-05-08T05:00:00Z Content type: tutorial Language: en Sources: [Blog RSS Feed | Snyk]() Topics: [Python](), [pip](), [Code](), [Security](), [TLS (Transport Layer Security)](), [GitHub]() Tags: [acquisition](), [awareness](), [blog](), [code](), [code-security](), [developer](), [devsecops](), [documentation](), [draftdotdev](), [engineering](), [github](), [guide](), [how-to](), [installation](), [open-source-security](), [packages](), [pycharm](), [pypi](), [python](), [python-packages](), [security](), [snyk-code](), [snyk-open-source](), [tls](), [tutorial]() ### AI overview This guide explains how to create a modern secure Python package, covering package structure, metadata, tests, documentation, installation, imports, distribution, and use of public or private package indexes. It also discusses using TLS to secure private indexes and provides a GitHub repository with the tutorial code. ### Source excerpt This guide will detail all the steps to build a modern Python package. ## What's new in TensorFlow 2.16 DevFeed: [What's new in TensorFlow 2.16]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2024-03-13T20:11:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [TensorFlow Core](), [Keras](), [Compiler](), [LLVM](), [Windows](), [pip](), [Python](), [MSVC]() Tags: [announcement](), [compiler](), [estimator](), [keras](), [llvm](), [msvc](), [python](), [release](), [tensorflow-core](), [windows]() ### AI overview TensorFlow 2.16 introduces Clang as the default compiler for TensorFlow CPU wheels on Windows, makes Keras 3 the default Keras version, and adds Python 3.12 support. The release also removes the tf.estimator API and changes the recommended installation method for Apple Silicon. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.16 has been released! Highlights of this release (and 2.15) include Clang as default compiler for building TensorFlow CPU wheels on Windows, Keras 3 as default version, support for Python 3.12, and much more! For the full release note, please click here. Note: Release updates on the new multi-backend Keras will be published on keras.io starting with Keras 3.0. For more information, please see https://keras.io/keras_3/. TensorFlow Core Clang 17 Clang is now the preferred compiler to build TensorFlow CPU wheels on the Windows Platform starting with this release. The currently supported version is LLVM/clang 17. The official Wheels-published on PyPI will be based on Clang; however, users retain the option to build wheels using the MSVC compiler following the steps mentioned, as has been the case before. Intel owned the implementation and delivery of this change within the 3P Official Build program. Keras 3 Keras 3 will be the default Keras version for TensorFlow 2.16 onwards. You may need to update your script to use Keras 3. Please refer to the new Keras documentation for Keras 3 (https://keras.io/keras_3). Keras 2 will continue to be released alongside TensorFlow as tf_keras. To continue using Keras 2 with TensorFlow 2.16+: Install tf-keras vía pip install tf-keras~=2.16 Switch tf.keras to use Keras 2 (tf-keras), by setting environment variable TF_USE_LEGACY_KERAS=1 directly or in your Python program by doing import os;os.environ["TF_USE_LEGACY_KERAS"]="1". Please note that this needs to be set before importing TensorFlow and will set it for all packages in your Python runtime program. Estimator API tf.estimator API is removed. If you need to use the estimator API, you need to use TF 2.15 or an earlier version. Apple Silicon If you previously installed TensorFlow using pip install tensorflow-macos, please update your installation method. Use pip install tensorflow from now on. tensorflow-macos package will no longer receive ## TensorFlow 2.15 update: hot-fix for Linux installation issue DevFeed: [TensorFlow 2.15 update: hot-fix for Linux installation issue]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2023-12-05T22:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [pip](), [Linux](), [TensorRT](), [CUDA](), [Python]() Tags: [announcement](), [dependencies](), [explore](), [installation](), [keras](), [linux](), [nvdia](), [nvidia](), [packages](), [release](), [tensorflow](), [tensorflow-core](), [tensorrt](), [update](), [x86-64]() ### AI overview TensorFlow released version 2.15.0.post1 to fix Linux x86_64 installation problems caused by missing TensorRT-related Python dependencies when installing TensorFlow 2.15 with NVIDIA CUDA dependencies via pip. The hotfix restores the intended installation behavior, while users pinning versions must specify 2.15.0.post1 explicitly. ### Source excerpt Posted by the TensorFlow team We are releasing a hot-fix for an installation issue affecting the TensorFlow installation process. The TensorFlow 2.15.0 Python package was released such that it requested tensorrt-related packages that cannot be found unless the user installs them beforehand or provides additional installation flags. This dependency affected anyone installing TensorFlow 2.15 alongside NVIDIA CUDA dependencies via pip install tensorflow[and-cuda]. Depending on the installation method, TensorFlow 2.14 would be installed instead of 2.15, or users could receive an installation error due to those missing dependencies. To solve this issue as quickly as possible, we have released TensorFlow 2.15.0.post1 for the Linux x86_64 platform. This version removes the tensorrt Python package dependencies from the tensorflow[and-cuda] installation method. Support for TensorRT is otherwise unaffected as long as TensorRT is already installed on the system. Now, pip install tensorflow[and-cuda] works as originally intended for TensorFlow 2.15. Using .post1 instead of a full minor release allowed us to push this release out quickly. However, please be aware of the following caveat: for users wishing to pin their Python dependency in a requirements file or other situation, under Python's version specification rules, tensorflow[and-cuda]==2.15.0 will not install this fixed version. Please use ==2.15.0.post1 to specify this exact version on Linux platforms, or a fuzzy version specification, such as ==2.15.*, to specify the most recent compatible version of TensorFlow 2.15 on all platforms. ## What's new in TensorFlow 2.15 DevFeed: [What's new in TensorFlow 2.15]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2023-11-17T18:55:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [TensorFlow Core](), [CUDA](), [cpu](), [Linux](), [pip](), [Windows](), [Compiler]() Tags: [announcement](), [compiler](), [cpu](), [cuda](), [explore](), [installation](), [keras](), [linux](), [nvdia](), [performance](), [release](), [tensorflow](), [tensorflow-core](), [windows]() ### AI overview TensorFlow 2.15 introduces simpler optional installation of NVIDIA CUDA libraries through pip on Linux, provided the NVIDIA driver is already installed. It also enables oneDNN CPU optimizations by default for Windows x64 and x86 packages, adds full tf.function type support, upgrades CUDA to 12.2, and uses Clang 17 for package builds and source compilation. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.15 has been released! Highlights of this release (and 2.14) include a much simpler installation method for NVIDIA CUDA libraries for Linux, oneDNN CPU performance optimizations for Windows x64 and x86, full availability of tf.function types, an upgrade to Clang 17.0.1, and much more! For the full release note, please check here. Note: Release updates on the new multi-backend Keras will be published on keras.io starting with Keras 3.0. For more information, please check here. TensorFlow Core NVIDIA CUDA libraries for Linux The tensorflow pip package has a new, optional installation method for Linux that installs necessary NVIDIA CUDA libraries through pip. As long as the NVIDIA driver is already installed on the system, you may now run pip install tensorflow[and-cuda] to install TensorFlow's NVIDIA CUDA library dependencies in the Python environment. Aside from the NVIDIA driver, no other pre-existing NVIDIA CUDA packages are necessary. In TensorFlow 2.15, CUDA has been upgraded to version 12.2. oneDNN CPU performance optimizations For Windows x64 & x86 packages, oneDNN optimizations are now enabled by default on X86 CPUs. These optimizations can be enabled or disabled by setting the environment variable TF_ENABLE_ONEDNN_OPTS to 1 (enable) or 0 (disable) before running TensorFlow. To fall back to default settings, simply unset the environment variable. tf.function tf.function types are now fully available. tf.types.experimental.TraceType now allows custom tf.function inputs to declare Tensor decomposition and type casting support. Introducing tf.types.experimental.FunctionType as the comprehensive representation of the signature of tf.function callables. It can be accessed through the function_type property of tf.function's and ConcreteFunctions. See the tf.types.experimental.FunctionType documentation for more details. Introducing tf.types.experimental.AtomicFunction as the fastest way to perform TF computations in Python. ## Using WSL and Let's Encrypt to create Azure App Service SSL Wildcard Certificates DevFeed: [Using WSL and Let's Encrypt to create Azure App Service SSL Wildcard Certificates]() Original publisher: [Read original article]() Author: Scott Hanselman Published: 2023-06-27T17:17:25Z Content type: tutorial Language: en Sources: [Scott Hanselman]() Topics: [Azure](), [Windows Subsystem for Linux](), [SSL](), [openssl](), [apt](), [pip]() Tags: [apt](), [azure](), [certificates](), [dns](), [install](), [openssl](), [password](), [private-key](), [ssl](), [windows](), [wsl]() ### AI overview This tutorial explains how to use Certbot in Windows Subsystem for Linux with Let's Encrypt to create a wildcard certificate, convert it to a password-protected PFX containing the certificate chain, and upload it to Azure App Service. It also describes importing and exporting the certificate in Windows before updating the App Service binding. ### Source excerpt There are many let's encrypt automatic tools for azure but I also wanted to see if I could use certbot in wsl to generate a wildcard certificate for the azure Friday website and then upload the resulting certificates to azure app service. Azure app service ultimately needs a specific format called dot PFX that includes the full certificate path and all intermediates. Per the docs, App Service private certificates must meet the following requirements: Exported as a password-protected PFX file, encrypted using triple DES. Contains private key at least 2048 bits long Contains all intermediate certificates and the root certificate in the certificate chain. If you have a PFX that doesn't meet all these requirements you can have Windows reencrypt the file. I use WSL and certbot to create the cert, then I import/export in Windows and upload the resulting PFX. Within WSL, install certbot: sudo apt update sudo apt install python3 python3-venv libaugeas0 sudo python3 -m venv /opt/certbot/ sudo /opt/certbot/bin/pip install --upgrade pip sudo /opt/certbot/bin/pip install certbot Then I generate the cert. You'll get a nice text UI from certbot and update your DNS as a verification challenge. Change this to make sure it's two lines, and your domains and subdomains are correct and your paths are correct. sudo certbot certonly --manual --preferred-challenges=dns --email YOUR@EMAIL.COM --server https://acme-v02.api.letsencrypt.org/directory --agree-tos --manual-public-ip-logging-ok -d "azurefriday.com" -d "*.azurefriday.com" sudo openssl pkcs12 -export -out AzureFriday2023.pfx -inkey /etc/letsencrypt/live/azurefriday.com/privkey.pem -in /etc/letsencrypt/live/azurefriday.com/fullchain.pem I then copy the resulting file to my desktop (check your desktop path) so it's now in the Windows world. sudo cp AzureFriday2023.pfx /mnt/c/Users/Scott/OneDrive/Desktop Now from Windows, import the PFX, note the thumbprint and export that cert. Import-PfxCertificate -FilePath "AzureFriday2023.pfx" - ## Application Security Roundup - September DevFeed: [Application Security Roundup - September]() Original publisher: [Read original article]() Author: Adam Published: 2022-09-30T00:00:00Z Content type: article Language: en Sources: [Shostack & Friends Blog]() Topics: [Application Security](), [Security](), [DevSecOps](), [supply-chain-security](), [pip](), [PyPI](), [Python](), [PyTorch](), [Cryptography]() Tags: [application-security](), [appsec](), [cryptography](), [devsecops](), [pypi](), [python](), [pytorch](), [security](), [supply-chain-security]() ### AI overview A monthly application security roundup covering access-control failures, STRIDE, DevSecOps, attack trees, software supply-chain attacks, customer-service risks, PyPI package installation behavior, PyTorch loading behavior, and cryptography. ### Source excerpt Interesting appsec posts: machine learning, performance, and C4 ## From YAGNI to YDNIY DevFeed: [From YAGNI to YDNIY]() Original publisher: [Read original article]() Author: Itamar Turner-Trauring Published: 2020-09-18T00:00:00Z Content type: tutorial Language: en Sources: [Itamar Turner-Trauring]() Topics: [Python](), [Conda](), [jupyter](), [Linux](), [macOS](), [pip](), [data](), [C++]() Tags: [c-plus-plus](), [conda](), [jupyter](), [jupyter-notebooks](), [linux](), [macos](), [python](), [releases]() ### AI overview The article introduces YDNIY, or "You Don't Need It Yet," as a complement to YAGNI. It argues that features that are necessary but not release blockers can be delayed when a product remains useful without them. The author illustrates this with the staged development of the Python memory profiler Fil across Linux, macOS, Jupyter, Conda, and multiple memory-allocation APIs. ### Source excerpt How do you ship a product on schedule? One useful approach is applying the You Ain't Gonna Need It principle, or YAGNI for short: leave out all the things that seem nice-to-have, but you have no proof you actually need. But beyond the things you don't need, there are still plenty of features you pretty clearly do need... but are not blockers on releasing your product. So beyond YAGNI, there's also YDNIY: You Don't Need It Yet. Let's see an example of this principle in practice, visualize the principle as a flowchart, and then compare it to another popular acronymed concept, the Minimum Viable Product. A real world example: shipping a new memory profiler In March 2020 I shipped the initial release of a new memory profiler for Python, Fil. Here's how it changed over time in terms of features, from May to August 2020: 0.3.0, initial release: Installable via pip packaging tool, runs only on Linux, only profiles complete program runs. 0.3.3: Support for an additional memory allocation API. 0.4.0: Support for out-of-memory situations. 0.5.0: macOS support. 0.6.0: Support for mmap() allocation API. 0.7.0: Support for C++ memory allocation API. 0.9.0: Much faster and lower overhead in some use cases, added support for yet another memory allocation API. 0.10.0: Support for running inside Jupyter notebooks, and native support for installing via the Conda packaging tool. All of the features I added in later releases were clearly necessary from the start; YAGNI did not apply. Lots of people use macOS, the target audience of data scientists and scientists often use Conda and Jupyter, all those memory allocation APIs are used in the real world, and so on. But even a tool that only runs complete programs on Linux, and only tracks the most popular memory allocation APIs, is still useful to some people. If I had waited until all those features were implemented to ship an initial release, all the people who used the profiler during the first four months of its existence would have had ## Installing Ansible on Ubuntu 18 on AWS DevFeed: [Installing Ansible on Ubuntu 18 on AWS]() Original publisher: [Read original article]() Author: Fuzzygroup Published: 2020-05-12T00:00:00Z Content type: tutorial Language: en Sources: [Scott Johnson]() Topics: [Ansible](), [Ubuntu](), [Amazon Web Services](), [configuration](), [pip](), [DevOps]() Tags: [ansible](), [apt](), [aws](), [commands](), [configuration](), [devops](), [install](), [installation](), [linux](), [open-source](), [python](), [server](), [ubuntu]() ### AI overview A practical guide to installing Ansible on Ubuntu 18 for AWS, covering the required repository, package, Python pip, and boto or boto3 installation commands. ### Source excerpt Ansible is an Open Source DevOps engine for systems configuration. Here is the process for installing Ansible on Ubuntu 18 for AWS. sudo apt-add-repository ppa:ansible/ansible sudo apt update sudo apt install ansible sudo apt install python-pip pip install boto3 --user pip install boto --user Note: The commands pip install boto3 -user / pip install boto -user - you only need one of these. Use boto3 if your Ansible installation is using python3 and boto if your Ansible is using python 2.x. As with a bunch of my shorter blog posts around installing software, this one is because I needed to install Ansible on a server for work and I figured that documenting it was likely useful (this is now the second time I've had to do it for this job and there will undoubtedly be a third). Sources Installing Ansible on Ubuntu Installing Boto on Linux ## Upgrading Python and Pip and VirtualEnv on Ubuntu DevFeed: [Upgrading Python and Pip and VirtualEnv on Ubuntu]() Original publisher: [Read original article]() Author: Fuzzygroup Published: 2020-04-03T00:00:00Z Content type: tutorial Language: en Sources: [Scott Johnson]() Topics: [Python](), [Ubuntu](), [pip](), [apt]() Tags: [apt](), [install](), [python](), [ubuntu](), [upgrade]() ### AI overview A brief tutorial on upgrading Python, pip, and virtualenv on Ubuntu after a Python package installation ends with a segmentation fault. ### Source excerpt I found myself this morning trying to get a Python stack running on an Ubuntu instance only to hit this crap: Installing collected packages: numpy, pytz, six, python-dateutil, pandas, nltk, pyparsing, cycler, kiwisolver, matplotlib, kafka-python, pyyaml, murmurhash, zipp, importlib-metadata, catalogue, plac, tqdm, setuptools, srsly, cymem, preshed, blis, wasabi, thinc, idna, urllib3, certifi, chardet, requests, spacy, pluggy, attrs, py, more-itertools, wcwidth, packaging, pytest, langdetect, spacy-langdetect, editorconfig, jsbeautifier, docutils, jmespath, botocore, s3transfer, boto3, torch, dataclasses, tokenizers, click, joblib, regex, sacremoses, filelock, sentencepiece, transformers, scipy, scikit-learn, gunicorn, WebOb, firefly-python, pycparser, cffi, zstandard, lefthook, termcolor, fire, redis, sseclient, absl-py, protobuf, markdown, pyasn1, pyasn1-modules, cachetools, rsa, google-auth, tensorboard-plugin-wit, werkzeug, grpcio, wheel, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, future, pytorch-lightning Segmentation fault (core dumped) So I guess it is time to do the Python upgrade dance. Here you go: pip3 install --upgrade pip sudo apt-get update && sudo apt-get upgrade sudo apt-get install python3.7 sudo pip3 install --upgrade virtualenv Stack Overflow References Pip upgrade. Python Upgrade ## Object Storage Use Cases Part 3: Storing binary files in Object Storage instead of in your git repository DevFeed: [Object Storage Use Cases Part 3: Storing binary files in Object Storage instead of in your git repository]() Original publisher: [Read original article]() Author: Unknown (noreply@blogger.com) Published: 2019-04-29T09:06:00Z Content type: tutorial Language: en Sources: [CSC - IT Center For Science - Cloud Team]() Topics: [Git](), [ceph](), [pip](), [Haskell]() Tags: [aws](), [ceph](), [data](), [data-lake](), [git](), [install](), [object-storage](), [python](), [rados](), [radosgw](), [s3](), [storage]() ### AI overview This post explains why binary files should not be stored directly in Git repositories and explores using CSC's Pouta Cloud Object Storage instead. It documents attempts with git-annex and git-bigstore, including a patch to configure a non-default S3-compatible endpoint. ### Source excerpt 1. It's bad to store binary blobs in git. This is known [1] 2. A great place to store binary files is in object storage 3. Can we use CSC's Pouta Cloud Object Storage service for this? Why yes, yes you can! This is a post in a series about different use cases for object storage. Others include for example hosting a reveal.js presentation and NFS server backups More details about CSC's Pouta Cloud Object Storage Service, it is based on CEPH Rados Object Storage Gateway. What The general solution* to storing binaries in git is to not store them in git and instead create a link to some other place Attempt 1 Install an application that you like. In https://www.perforce.com/blog/storing-large-binary-files-in-git-repositories I searched for "S3", found git-annex which had recent commits so it looked like it wasn't dead. Installation instructions: https://git-annex.branchable.com/install/ The version in EPEL/Ubuntu Xenial does not have S3 support and is from 2014. There's a recent one in Fedora's repos though. Unfortunately, compiling it requires cabal. I don't know Haskell. https://spack.readthedocs.io/en/latest/package_list.html does not have any Haskell, cabal or git-annex available. Next. Attempt 2 Next application in the blog post was https://github.com/lionheart/git-bigstore To install it was much more comfortable for me: pip install "git-bigstore<=2.0" First issue was that looking at https://github.com/lionheart/git-bigstore/blob/master/bigstore/backends/s3.py there was no way to tell it to use another endpoint than the default of AWS S3. A small patch takes care of that, add , endpoint_url='https://object.pouta.csc.fi:443' to the client() boto3 call in the backends/s3.py file from git-bigstore. For a more detailed example see this Pull Request. After spending some time going down some rabbit holes it seems that the Profile option that is supported in git-bigstore does not support endpoints. It can only be used to specify credentials and a few other parameters. Mino ## Getting Started with Pipenv DevFeed: [Getting Started with Pipenv]() Original publisher: [Read original article]() Author: masnun Published: 2017-11-25T23:09:32Z Content type: tutorial Language: en Sources: [Abu Ashraf Masnun]() Topics: [Python](), [pip](), [Package manager](), [PyPI](), [Flask](), [PyCharm](), [REST API](), [Windows]() Tags: [dependencies](), [environment-management](), [getting-started](), [install](), [packaging](), [pycharm](), [pypi](), [python](), [python-packages](), [rest-api](), [windows]() ### AI overview A tutorial introducing Pipenv for Python development. It explains how Pipenv combines package installation with isolated virtual environments, records dependencies in a Pipfile, creates a lock file for deterministic builds, and supports installing packages, running applications, activating environments, and managing dependencies. ### Source excerpt If you're a Python developer, you probably know about pip and the different environment management solutions like virtualenv or venv. The pip tool is currently the standard way to install a Python package. Virtualenv has been a popular way of isolating Python environments for a long time. Pipenv combines the very best of these tools [...] The post Getting Started with Pipenv appeared first on Abu Ashraf Masnun. ## Python Unit Testing with Mocks: Patching and Namespaces DevFeed: [Python Unit Testing with Mocks: Patching and Namespaces]() Original publisher: [Read original article]() Author: Anjana Sofia Vakil (contact@anjana.dev) Published: 2016-06-16T00:00:00Z Content type: tutorial Language: en Sources: [Mozilla Automation]() Topics: [Mocking](), [Python](), [Testing](), [Library](), [PyPI](), [pip]() Tags: [install](), [library](), [mocking](), [patches](), [pypi](), [python](), [testing](), [unit-tests]() ### AI overview A tutorial on using Python mocks in unit tests to isolate specific behavior, inspect method calls, simulate return values, and patch objects in the correct module namespace. ### Source excerpt This post brought to you from Mozilla's London All Hands meeting - cheers! When writing Python unit tests, sometimes you want to just test one specific aspect of a piece of code that does multiple things. For example, maybe you're wondering: Does object X get created here? Does method X get called here? Assuming method X returns Y, does the right thing happen after that? Finding the answers to such questions is super simple if you use mock: a library which "allows you to replace parts of your system under test with mock objects and make assertions about how they have been used." Since Python 3.3 it's available simply as unittest.mock, but if you're using an earlier Python you can get it from PyPI with pip install mock. So, what are mocks? How do you use them? Well, in short I could tell you that a Mock is a sort of magical object that's intended to be a doppelgänger for some object in your code that you want to test. Mocks have special attributes and methods you can use to find out how your test is using the object you're mocking. For example, you can use Mock.called and .call_count to find out if and how many times a method has been called. You can also manipulate Mocks to simulate functionality that you're not directly testing, but is necessary for the code you're testing. For example, you can set Mock.return_value to pretend that an function gave you some particular output, and make sure that the right thing happens in your program. But honestly, I don't think I could give a better or more succinct overview of mocks than the Quick Guide, so for a real intro you should go read that. While you're doing that, I'm going to watch this fantastic Michael Jackson video: Oh you're back? Hi! So, now that you have a basic idea of what makes Mocks super cool, let me share with you some of the tips/tips/trials/tribulations I discovered when starting to use them. Patches and namespaces tl;dr: Learn where to patch if you don't want to be sad! When you import a helper module int ## pip - uninstall package with dependencies DevFeed: [pip - uninstall package with dependencies]() Original publisher: [Read original article]() Author: Tom Published: 2016-04-26T00:00:00Z Content type: tutorial Language: en Sources: [Tomasz Gągor]() Topics: [pip](), [Scripting, bash](), [Python]() Tags: [bash](), [dependencies](), [install](), [python]() ### AI overview A Bash snippet recursively uninstalls a pip package and its dependencies, with a note that sudo may be needed depending on how the package was installed. ### Source excerpt Virtualenvs in python are cheap but from time to time you will install something with pip on your system and when time comes removing all this crap could be difficult. I found this bash snippet1 that will uninstall package with all dependencies: Recursive pip uninstall for dep in $(pip show python-neutronclient | grep Requires | sed 's/Requires: //g; s/,//g') ; do pip uninstall -y $dep ; done pip uninstall -y python-neutronclient Depending how you installed it, you might need to use sudo. ## Using Google Data APIs with django apps DevFeed: [Using Google Data APIs with django apps]() Original publisher: [Read original article]() Published: 2014-06-07T00:00:00Z Content type: tutorial Language: en Sources: [HackerEarth]() Topics: [Google](), [API](), [Django](), [Authentication](), [Python](), [Library](), [pip]() Tags: [apis](), [authentication](), [django](), [google](), [library](), [python]() ### AI overview This tutorial explains how to integrate Google Data APIs into a Django application using the GData Python client library. It covers creating a Google project, enabling APIs, configuring credentials and redirect URIs, authenticating users, storing tokens in the session, and making API calls such as retrieving contacts. ### Source excerpt ####Setting up a Google Project In order to use any of the Google APIs for your application, first you need to set up a project in the Google Developer's Console. Enable all the APIs that you want to use in the APIs tab under APIs and auth. Under the Credentials tab, create a Client ID and Client secret which is used for communication between your application and the API. Enter all the allowed redirect urls in the Redirect URIs field. These are the URLs to which the application redirects after a user is successfully authenticated. You can change these URLs any time you want. Now your Google App is ready for use. We wanted to create a small application where users can invite their google contacts to join HackerEarth. ####Dependencies We use the GData python client library, which makes it easy to interact with these Google services. You can install it using pip: sudo pip install gdata or install it from [source](https://code.google.com/p/gdata-python-client/downloads/list). ####Authentication First we need to define some constants that we will use througout the application #Obtained from Google Project Settings GOOGLE_CLIENT_ID = GOOGLE_CLIENT_SECRET = #Variable that specifies the data you want to access GOOGLE_SCOPE = "http(s)://www.google.com/m8/feeds/" #URL where the flow should go on successful authentication GOOGLE_APPLICATION_REDIRECT_URI = GOOGLE_REDIRECT_SESSION_VAR = Since we wanted to use the Contacts API we used the above mentioned scope. A list of other scopes is given here. The first step for authentication is generation of an authentication token. Since we have written a separate view to handle the auth token after successful login, we are setting the authentication token in the session so that the same token can be accessed in both the views. #Try to fetch the authentication token from the session auth_token = request.session.get('google_auth_token') #If an authentication token does ## Installing Python Packages DevFeed: [Installing Python Packages]() Original publisher: [Read original article]() Author: Map Published: 2011-11-01T20:55:56Z Content type: tutorial Language: en Sources: [Craig Kerstiens]() Topics: [Python](), [pip](), [PyPI](), [Django]() Tags: [install](), [pypi](), [python](), [python-packages](), [requirements](), [version]() ### AI overview A tutorial on installing Python packages with pip, using a virtualenv environment, and recording installed package versions for sharing through a requirements list. It explains that packages are hosted on PyPI and demonstrates creating, activating, and deactivating an isolated project environment. ### Source excerpt Now that you have you system and project environment all setup you probably want to start developing. But you likely don't want to start writing an entire project fully from scratch, as you dive in you'll quickly realize theres many tools helping you build projects and sites faster. For example making a request to a website there's Requests, for handling processing images there's Python Imaging Library, or for a full framework to help you in building a site there's Django. With all of these there's one simple and common way to install them. But first a little more on how it all works. All major Python packages are hosted on PyPi (Pronounced Pi-P or Cheeseshop). When you use a common python installer it will: Search for the package you specify If you specify a version will use it, otherwise will use the latest Will download the source for that package Install it into your Python environment Now for actually installing... Lets get started with installing the three packages below. At this point you should at least have a fresh Python environment, however you don't have an immediate way to install packages. The defacto Python package installer is pip. Earlier we setup virtualenv to help isolate our python packages we were working with. First lets go ahead and create a folder for our project then setup a new environment for the project we'll work on: $ mkdir myapp $ cd myapp $ virtualenv --no-site-packages venv If we list the contents of the directory you'll now see a folder venv. Within this folder you'll find all the parts of the environment that virtualenv just created: $ ls venv $ ls venv bin include lib Now you've got a sandboxed environment that exists but you haven't loaded it. You can now activate and deactivate this any time you like. Once you do this it customizes your path to use the packages you've installed for this environment. To load your environment when in the myapp directory: $ source venv/bin/activate To deactivate this simple: $ deactivate Now that we' ## Contributing to Django: quickstart DevFeed: [Contributing to Django: quickstart]() Original publisher: [Read original article]() Author: Graham King Published: 2011-07-07T05:01:55Z Content type: tutorial Language: en Sources: [Graham King]() Topics: [Django](), [Development](), [pip](), [SQLite](), [MySQL](), [PostgreSQL]() Tags: [developers](), [development](), [development-environment](), [django](), [guide](), [how-to](), [install](), [mysql](), [patches](), [postgresql](), [python](), [software](), [sqlite](), [tests](), [version]() ### AI overview A step-by-step guide to setting up a development environment for contributing to Django core. It covers installing pip and virtualenv, installing dependencies, running the test suite, creating patches, and keeping a checkout current. ### Source excerpt "Empowering Django developers: A step-by-step guide to setting up your environment."