Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task]: Evaluate pushing from Oracle to Postgres #1388

Closed
1 task
acouch opened this issue Mar 1, 2024 · 3 comments
Closed
1 task

[Task]: Evaluate pushing from Oracle to Postgres #1388

acouch opened this issue Mar 1, 2024 · 3 comments
Assignees
Labels
project: grants.gov Grants.gov Modernization tickets

Comments

@acouch
Copy link
Collaborator

acouch commented Mar 1, 2024

Summary

There is a proposal to push form Oracle to Postgres instead of using the foreign data wrapper. There are a few benefits of having this synchronization process within the MH environment:

Simplified Security: removes external access into the Grants.gov Production environment
Less work on the Simpler side: by synchronizing the data internally into Aurora we can use logical replication to automatically push table changes to their database (AWS blocks this ability on Oracle)
Better logging on the production side: we have better visibility into the metrics and can adjust for performance on a per table basis
We can start with this simple synch for a small number of tables, then simply switch to DMS if the need increases, implementing it as a replication (push) process rather than pulling data externally

Acceptance criteria

  • evaluate proposal and comment in this ticket
@acouch acouch added the project: grants.gov Grants.gov Modernization tickets label Mar 1, 2024
@acouch acouch added this to the Search API - ELT Decision milestone Mar 1, 2024
@acouch
Copy link
Collaborator Author

acouch commented Mar 26, 2024

This is still on progress for this sprint. Working on #1299 first

@bretthrosenblatt
Copy link
Collaborator

The reason I suggested this approach is because it makes it much easier for Simpler to replicate the rest of grants.gov much more easily. It's much harder to migrate from the outside.

@acouch
Copy link
Collaborator Author

acouch commented Apr 16, 2024

We discussed this this morning and decided to focus on the pull approach as we own that part of the pipeline but can revisit in the future.

@acouch acouch closed this as completed Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
project: grants.gov Grants.gov Modernization tickets
Projects
Development

No branches or pull requests

2 participants