Skip to main content

Firebase Realtime Database

Availability:
Not available Airbyte CloudAvailable Airbyte OSSAvailable PyAirbyte
Support Level:
Marketplace
Connector Version:
0.1.27(Last updated 19 days ago)
CDK Version:
0.90.0
Sync Success Rate:
Usage Rate:

Overview​

The Firebase Realtime Database source supports Full Refresh sync. As the database data is stored as JSON objects and there are no records or tables, you can sync only one stream which you specifed as a JSON node path on your database at a time.

Resulting schema​

As mentioned above, fetched data is just a JSON objects. The resulting records conformed of two columns key and value. The key's value is keys (string) of fetched JSON object. The value's value is string representation of values (string representation of any JSON object) of fetched JSON object.

If your database has data as below at path https://{your-database-name}.firebaseio.com/store-a/users.json ...

{
  "liam": { "address": "somewhere", "age": 24 },
  "olivia": { "address": "somewhere", "age": 30 }
}

and you specified a store-a/users as a path in configuration, you would sync records like below ...

{"key": "liam", "value": "{\"address\": \"somewhere\", \"age\": 24}}"}
{"key": "olivia", "value": "{\"address\": \"somewhere\", \"age\": 30}}"}

Features​

FeatureSupportedNotes
Full Refresh SyncYes
Incremental SyncNo
Change Data CaptureNo
SSL SupportYes

Getting started​

Requirements​

To use the Firebase Realtime Database source, you'll need:

  • A Google Cloud Project with Firebase enabled
  • A Google Cloud Service Account with the "Firebase Realtime Database Viewer" roles in your Google Cloud project
  • A Service Account Key to authenticate into your Service Account

See the setup guide for more information about how to create the required resources.

Service account​

In order for Airbyte to sync data from Firebase Realtime Database, it needs credentials for a Service Account with the "Firebase Realtime Database Viewer" roles, which grants permissions to read from Firebase Realtime Database. We highly recommend that this Service Account is exclusive to Airbyte for ease of permissioning and auditing. However, you can use a pre-existing Service Account if you already have one with the correct permissions.

The easiest way to create a Service Account is to follow Google Cloud's guide for Creating a Service Account. Once you've created the Service Account, make sure to keep its ID handy as you will need to reference it when granting roles. Service Account IDs typically take the form <account-name>@<project-name>.iam.gserviceaccount.com

Then, add the service account as a Member in your Google Cloud Project with the "Firebase Realtime Database Viewer" role. To do this, follow the instructions for Granting Access in the Google documentation. The email address of the member you are adding is the same as the Service Account ID you just created.

At this point you should have a service account with the "Firebase Realtime Database" product-level permission.

Service account key​

Service Account Keys are used to authenticate as Google Service Accounts. For Airbyte to leverage the permissions you granted to the Service Account in the previous step, you'll need to provide its Service Account Keys. See the Google documentation for more information about Keys.

Follow the Creating and Managing Service Account Keys guide to create a key. Airbyte currently supports JSON Keys only, so make sure you create your key in that format. As soon as you created the key, make sure to download it, as that is the only time Google will allow you to see its contents. Once you've successfully configured Firebase Realtime Database as a source in Airbyte, delete this key from your computer.

Setup the Firebase Realtime Database source in Airbyte​

You should now have all the requirements needed to configure Firebase Realtime Database as a source in the UI. You'll need the following information to configure the Firebase Realtime Database source:

  • Database Name
  • Service Account Key JSON: the contents of your Service Account Key JSON file.
  • Node Path [Optional]: node path in your database's data which you want to sync. default value is ""(root node).
  • Buffer Size [Optional]: number of records to fetch at one time (buffered). default value is 10000.

Once you've configured Firebase Realtime Database as a source, delete the Service Account Key from your computer.

Reference​

Config fields reference

Field
Type
Property name
string
database_name
string
google_application_credentials
string
path
number
buffer_size

Changelog​

Expand to review
VersionDatePull RequestSubject
0.1.272024-11-2548653Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
0.1.262024-11-0447041Update dependencies
0.1.252024-10-1246799Update dependencies
0.1.242024-10-0546457Update dependencies
0.1.232024-09-2846135Update dependencies
0.1.222024-09-2145804Update dependencies
0.1.212024-09-1445505Update dependencies
0.1.202024-09-0745272Update dependencies
0.1.192024-08-3145055Update dependencies
0.1.182024-08-2444674Update dependencies
0.1.172024-08-1744299Update dependencies
0.1.162024-08-1243795Update dependencies
0.1.152024-08-1043600Update dependencies
0.1.142024-08-0343092Update dependencies
0.1.132024-07-2742609Update dependencies
0.1.122024-07-2042260Update dependencies
0.1.112024-07-1341900Update dependencies
0.1.102024-07-1041469Update dependencies
0.1.92024-07-0640816Update dependencies
0.1.82024-06-2940628Update dependencies
0.1.72024-06-2640538Update dependencies
0.1.62024-06-2540328Update dependencies
0.1.52024-06-2240181Update dependencies
0.1.42024-06-0639200[autopull] Upgrade base image to v1.2.2
0.1.32024-06-0338910Replace AirbyteLogger with logging.Logger
0.1.22024-06-0338910Replace AirbyteLogger with logging.Logger
0.1.12024-05-2038416[autopull] base image + poetry + up_to_date
0.1.02022-10-16#18029🎉 New Source: Firebase Realtime Database.
Was this page helpful?