Skip to content
View luc-pimentel's full-sized avatar
5 followers · 3 following

Highlights

  • Pro

Block or report luc-pimentel

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
luc-pimentel/README.md

Lucas Pimentel

Data engineer based in Brazil. Pipelines, the APIs around them, and the scrapers that get the data in the first place.

Open source

Contributions to the open-source data stack I use every day.

Some things I've built

  • YT_crawler: a Python library for YouTube search, video details, comments, transcripts and trending, without the official API. Typed results, with tests.
  • api_crawler: one interface over 11 data sources (Reddit, YouTube, GitHub, Indeed, Glassdoor and more) that saves raw JSON into your own data lake.
  • api-to-pydantic: a Claude agent that turns a JSON response, API URL or curl command into typed Pydantic models.

Pinned Loading

  1. apache/airflow apache/airflow Public

    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

    Python 47.4k 18.3k

  2. dbt-labs/dbt dbt-labs/dbt Public

    dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

    Rust 14.1k 2.7k

  3. infra-docs-rag infra-docs-rag Public

    RAG service over Kubernetes, Prometheus and Argo CD docs: cited answers, evals, observability. Work in progress.

    Python

  4. YT_crawler YT_crawler Public

    Scrape YouTube videos, channels, playlists, comments, and analytics data with this comprehensive crawling toolkit. Designed for data collection, research, and building YouTube-based applications.

    Python 6

  5. api_crawler api_crawler Public

    Aggregating APIs and web scrapers to easily gather public web data and build your own data lakes.

    Python 2