Destinations cdk: implement snapshotting spec test

[edgao]

tl;dr add a test that:

• runs spec on the connector
• diffs that spec against the spec in expected_spec.json
• writes that spec back out to expected_spec.json

the original spec.json got deleted at some point in this stack? so I just created a new expected_spec.json - no strong opinion on where to put it, but I wanted it to be more obvious that it's not the "canonical" spec.

other things to note:

• junit isn't able to discover the test from inside cdk/testFixtures, hence the class E2eSpecTest: SpecTest(). I'm pretty sure there's a way to make this work but haven't bothered looking into it yet, and I think this is good enough for now.
• SpecTest actually writes out the new expected_spec.json for convenience. The expectation is that we'll just git commit that anytime we modify the spec.
• jackson's pretty printer doesn't behave the same as prettier, so I excluded the expected_spec.json files from airbyte-ci's format command.

I already committed the updated spec, but there's still a few diffs against the original spec. Pasting them here, I think they're all intentional (e.g. changing type, tweaking wording, swapping example -> examples)

The property "$.connectionSpecification.properties.test_destination.oneOf.0.properties.test_destination_type.const" in the expected json is not found
The property "$.connectionSpecification.properties.test_destination.oneOf.0.properties.logging_config.oneOf.1.properties.max_entry_count.description" didn't match. Expected "Max number of entries to log. This destination is for testing only. So it won't make sense to log infinitely. The maximum is 1,000 entries.", Received: "Number of entries to log. This destination is for testing only. So it won't make sense to log infinitely. The maximum is 1,000 entries."
The property "$.connectionSpecification.properties.test_destination.oneOf.0.properties.logging_config.oneOf.1.properties.nth_entry_to_log.example" in the expected json is not found
The property "$.connectionSpecification.properties.test_destination.oneOf.0.properties.logging_config.oneOf.1.properties.nth_entry_to_log.type" didn't match. Expected "number", Received: "integer"
The property "$.connectionSpecification.properties.test_destination.oneOf.0.properties.logging_config.oneOf.2.properties.max_entry_count.description" didn't match. Expected "Max number of entries to log. This destination is for testing only. So it won't make sense to log infinitely. The maximum is 1,000 entries.", Received: "Number of entries to log. This destination is for testing only. So it won't make sense to log infinitely. The maximum is 1,000 entries."
The property "$.connectionSpecification.properties.test_destination.oneOf.1.properties.test_destination_type.const" in the expected json is not found
The property "$.connectionSpecification.properties.test_destination.oneOf.2.properties.test_destination_type.const" in the expected json is not found
The property "$.connectionSpecification.properties.test_destination.oneOf.2.properties.millis_per_record.description" didn't match. Expected "Number of milli-second to pause in between records.", Received: "The number of milliseconds to wait between each record."
The property "$.connectionSpecification.properties.test_destination.oneOf.3.properties.test_destination_type.const" in the expected json is not found

---

example diff in expected_spec.json:

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 10:00pm

[edgao]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Bulk Load CDK: Add integration test using in-memory mock destination #45634
• Destinations cdk: implement snapshotting spec test #45468 👈
• airbyte-ci: Ignore autogen files #45632
• Destinations CDK: better integration tests #45113
• Bulk CDK: Add conveniencen method to BufferingOutputConsumer to track message consumption #45611
• Bulk CDK (Destinations): Use Descriptor instead of full Stream as map key #45610
• Destinations cdk: recorddiffer that handles strongly-typed records #44874
• Load CDK E2ETest Destination w/o tests #45147
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @edgao and the rest of your teammates on Graphite

[edgao]

build is broken, commented this out just for testing purposes