Create dbt converter for Airbyte Hubspot source connector

[marcosmarxm]

The goal of this task is to ensure the Airbyte Hubspot connector output can be used to run Fivetran's dbt analytical model.

Definition of Done:

• Create a pull request that adds new tables or views, enabling the execution of Fiventran's dbt with Airbyte output data.
• Update the documentation inside the connector folder to explain how to configure the user dbt package to run the analytical model.

Resources:

• Check example of how it was done for Zendesk Support connector

[walkoss]

Hey @marcosmarxm
Could you assign it to me? :)

[walkoss]

Hey @marcosmarxm
Here is the detail for Step1

Fivetran Table	Airbyte Table
calendar_event	❌ Doesn't exists
company	companies
company_property_history	companies_property_history
contact_merge_audit	contacts_merged_audit
contact	contacts
contact_form_submission	contacts_form_submissions
contact_list	contact_lists
contact_list_member	contacts_list_memberships
contact_property_history	contacts_property_history
deal	deals
deal_stage	No direct match; integrated into deals
deal_company	No direct match; integrated into deals
deal_contact	No direct match; integrated into deals
deal_pipeline	deal_pipelines
deal_pipeline_stage	No direct match; integrated into deal_pipelines
deal_property_history	deals_property_history
email_campaign	No direct match; integrated into campaigns
email_event	email_events
email_event_bounce	No direct match; integrated into email_events
email_event_click	No direct match; integrated into email_events
email_event_deferred	No direct match; integrated into email_events
email_event_delivered	No direct match; integrated into email_events
email_event_dropped	No direct match; integrated into email_events
email_event_forward	No direct match; integrated into email_events
email_event_open	No direct match; integrated into email_events
email_event_print	No direct match; integrated into email_events
email_event_sent	No direct match; integrated into email_events
email_event_spam_report	No direct match; integrated into email_events
email_event_status_change	No direct match; integrated into email_events
email_subscription	email_subscriptions
email_subscription_change	subscription_changes
engagement	engagements
engagement_call	engagements_calls
engagement_company	No direct match; integrated into engagements
engagement_contact	No direct match; integrated into engagements
engagement_deal	No direct match; integrated into engagements
engagement_email	engagements_emails
engagement_email_cc	No direct match; integrated into engagements_emails
engagement_email_to	No direct match; integrated into engagements_emails
engagement_meeting	engagements_meetings
engagement_note	engagements_notes
engagement_task	engagements_tasks
form	forms
owner	owners
property	❌ Doesn't exists
property_option	❌ Doesn't exists
ticket_company	No direct match; integrated into tickets
ticket_contact	No direct match; integrated into tickets
ticket_deal	No direct match; integrated into tickets
ticket_engagement	No direct match; integrated into engagements
ticket_pipeline	ticket_pipelines
ticket_pipeline_stage	No direct match; integrated into ticket_pipelines
ticket_property_history	No direct match; may be integrated into tickets
ticket	tickets
merged_deal	No direct match; may be integrated into deals

Notes:
Some tables from Airbyte couldn't be found on Fivetran dbt package

• cars
• cars_web_analytics
• companies_web_analytics
• contacts_web_analytics
• deals_archived
• deals_web_analytics
• engagements_calls_web_analytics
• engagements_emails_web_analytics
• engagements_meetings_web_analytics
• engagements_notes_web_analytics
• engagements_tasks_web_analytics
• form_submissions
• goals
• goals_web_analytics
• line_items
• line_items_web_analytics
• owners_archived
• pets
• pets_web_analytics
• products
• products_web_analytics
• tickets_web_analytics
• workflows

What do you suggest for those unexisting streams that are on Airbyte?

[marcosmarxm]

@walkoss they won't be used by Fivetran so not a big issue.
The main issue can happen with calendar and the property table. It looks at least for properties is only a missing yaml file from our side https://docs.airbyte.com/integrations/sources/hubspot#step-2-configure-the-scopes-for-your-streams the connector supports I'll take a look and return soon to you! Thanks for the first step!

[walkoss]

@walkoss they won't be used by Fivetran so not a big issue. The main issue can happen with calendar and the property table. It looks at least for properties is only a missing yaml file from our side https://docs.airbyte.com/integrations/sources/hubspot#step-2-configure-the-scopes-for-your-streams the connector supports I'll take a look and return soon to you! Thanks for the first step!

@marcosmarxm Where do you see the property stream on Airbyte?

There are:

• Contacts Property History
• Companies Property History
• Deals Property History

But those are already handled by respectively contact_property_history, company_property_history and deal_property_history

[marcosmarxm]

Sorry the missing update @walkoss it looks Airbyte has property history but it was missing in the credential I used to generate the schema. Check here: https://docs.airbyte.com/integrations/sources/hubspot#notes-on-the-property_history-streams

[marcosmarxm]

@walkoss are you in Airbyte Slack? Feel free to ping me there if you have any question.

[btkcodedev]

Hi, this space looks inactive for long, can I work on this issue if possible?

[btkcodedev]

@marcosmarxm :D

[walkoss]

Hello @btkcodedev
I let you this one, I have started step1 and you can continue from that