Configuration

To set up the GitHub source, provide the following configuration parameters:

KeyDescriptionSample ValueRequired
access_tokenA GitHub access token containing credentials to access your desired repository.abcdefghijklmnopqrstuvwxyz1234567890ABCD
repositoryThe repository path (everything after github.com).mage-ai/mage-ai
start_dateA sync start date. Format: ISO8601 (YYYY-MM-DDTHH:MM:SSZ).2021-01-01T00:00:00Z
request_timeoutRequest timeout value in seconds.300
base_urlThe base URL before the repository path (defaults to https://api.github.com).https://api.github.com

Generating Credentials

Follow this guide to generate a Personal Access Token (PAT).


Supported Endpoints

This source pulls raw data from the GitHub REST API and extracts the following resources for a single repository:

  • Assignees
  • Collaborators
  • Commits
  • Commit Comments
  • Events
  • Issues
  • Issue Events
  • Issue Milestones
  • Projects
  • Project Cards
  • Project Columns
  • Pull Requests
  • PR Commits
  • Releases
  • Comments
  • Reviews
  • Review Comments
  • Stargazers
  • Teams
  • Team Members
  • Team Memberships

It outputs the schema for each resource.

Note:
Incremental sync is currently not supported for any endpoint.
To add support, the IncrementalStream class in ./tap_github/streams.py needs modifications to correctly handle bookmark values.