Wrangles

Full documentation available at wrangles.io.

Local development

Supported local development uses Python 3.13. On Windows, create or refresh the complete test and tooling environment with one command from the repository root:

.\scripts\bootstrap-dev.ps1

Add -RunTests to run the self-contained local test suite after installation. The script creates .venv, installs requirements-dev.txt and this checkout in editable mode, runs pip check, and verifies representative core and SQL connector imports. It will not replace an existing environment created with another Python version.

If .venv was created with an older Python version and can be discarded, deactivate it and recreate it from the committed declaration:

deactivate
Remove-Item -LiteralPath .\.venv -Recurse -Force
.\scripts\bootstrap-dev.ps1 -RunTests
.\.venv\Scripts\Activate.ps1

Dev Containers and Codespaces use the same Python 3.13 developer declaration automatically. pytest-local.ini is the shared credential-safe test selection. scripts/test-local.ps1 clears credential variables before running it and isolates generated pytest state by Windows identity. The configuration excludes the intentionally live database, AWS, WrangleWorks, AI, and search-provider checks. The complete credentialed suite remains a CI validation and a local dependency/import pass does not claim live-service validation.

What are Wrangles?

Wrangles are a set of modular transformations for data cleaning and enrichment. Each Wrangle is optimized for a particular job, many of which are backed by sophisticated machine learning models.

With Wrangles, you can:

• Extract information from a set of messy descriptions.
• Predict which category items belong to.
• Standardize text data to a desired format.
• Move data from one system to another.
• Much more...

Wrangles are system independent, and allow you to pull data from one system, transform it and push it to another. Wrangles can be incorporated directly into python code, or an automated sequence of wrangles can be run as a recipe.

Installation

The python package can be installed using pip.

pip install wrangles

This installs the core package, which covers the vast majority of use cases: all data wrangles, recipe execution, Excel and CSV file I/O, HTTP connectors, and SQLite.

Optional dependencies

Connectors for databases, cloud storage, and external services require additional packages. Install only the ones you need:

Capability	Install
Microsoft SQL Server	pip install pymssql sqlalchemy
Microsoft Access	pip install pyodbc
DuckDB	pip install duckdb
PostgreSQL	pip install psycopg2-binary sqlalchemy
MySQL	pip install pymysql sqlalchemy
MongoDB	pip install pymongo
AWS S3	pip install boto3
Salesforce	pip install simple-salesforce
SFTP / SSH	pip install fabric
Notifications	pip install apprise
OpenAI SDK	pip install openai
Google Gemini	pip install google-generativeai
SerpAPI (web search)	pip install serpapi

If a connector is used without its required package installed, Wrangles will raise a clear ImportError with the exact pip install command needed.

Once installed, import the package into your code.

import wrangles

Authentication

Some Wrangles use cloud based machine learning models. To use them a WrangleWorks account is required.

Create a WrangleWorks account: Register

There are two ways to provide the credentials:

Environment Variables

The credentials can be saved as the environment variables:

• WRANGLES_USER
• WRANGLES_PASSWORD

Method

The credentials can be provided within the python code using the authenticate method, prior to calling other functions.

wrangles.authenticate('<user>', '<password>')

Usage

Functions

Wrangles can be used as functions, directly incorporated into python code.

Wrangles broadly accept a single input string, or a list of strings. If a list is provided, the results will be returned in an equivalent list in the same order and length as the original.

# Extract alphanumeric codes from a free text strings - e.g. find all part numbers in a set of product description
>>> import wrangles

>>> wrangles.extract.codes('replacement part ABCD1234ZZ')
['ABCD1234ZZ']

>>> wrangles.extract.codes(['replacement part ABCD1234ZZ', 'NNN555BBB this one has two XYZ789'])
[
    ['ABCD1234ZZ'],
    ['NNN555BBB', 'XYZ789']
]

Recipes

Recipes are written in YAML and allow a series of Wrangles to be run as an automated sequence.

Recipes can be triggered either from python code or a terminal command.

Run

# PYTHON
import wrangles
wrangles.recipe.run('recipe.wrgl.yml')

# TERMINAL
wrangles.recipe recipe.wrgl.yml

Recipe

# file: recipe.wrgl.yml
# ---
# Convert a CSV file to an Excel file
# and change the case of a column.
read:
  - file:
      name: file.csv

wrangles:
  - convert.case:
      input: my column
      case: upper

write:
  - file:
      name: file.xlsx