Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acouch patch 1 #2174

Closed
wants to merge 60 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
8c62178
Update internal_task.yml
acouch May 14, 2024
90d14d6
Delete .github/ISSUE_TEMPLATE/internal_30k.md
acouch May 14, 2024
a9ea8b6
Delete .github/ISSUE_TEMPLATE/internal_adr.yml
acouch May 14, 2024
6bfd75e
[Issue #1757]: setup e2e tests for search (#1)
rylew1 May 14, 2024
6944695
Merge branch 'HHS:main' into main
coilysiren May 14, 2024
493fb4e
[Issue #37]: finish e2e tests (#38)
rylew1 May 22, 2024
cd18a47
[Issue #1957]: sortby posted date desc default (#4)
rylew1 May 22, 2024
16f708e
Upgrade dependencies for API (May 21, 2024) (#48)
chouinar May 22, 2024
25b0295
[Issue #9] Setup opensearch locally (#39)
chouinar May 22, 2024
b40344d
[Issue #12] Setup the opportunity v1 endpoint which will be backed by…
chouinar May 22, 2024
879e743
[Issue #10] Populate the search index from the opportunity tables (#47)
chouinar May 22, 2024
2572fe0
Move Pages to App Router (#7)
acouch May 23, 2024
9b8cd17
[Issue #37]: e2e docs update (#49)
rylew1 May 24, 2024
3eb5fb5
[Issue #50]: change SortBy to USWDS component (#52)
rylew1 May 28, 2024
4047895
[Issue 56]: Date rounding bug (#57)
rylew1 May 31, 2024
0f38c92
[Issue #58] Locally, preserve the auth token in the OpenAPI across re…
chouinar Jun 3, 2024
9574e29
[Issue #51]: debounce pagination (#53)
rylew1 Jun 3, 2024
f12d0dc
[Issue #64] update to Next 14.2.3 (#65)
rylew1 Jun 4, 2024
2db4d35
[Issue #69] Remove the BASE_RESPONSE_SCHEMA (#70)
chouinar Jun 7, 2024
ddf0d75
[Task]: Finish adding Postgres Integration to Analytics Library (#72)
aplybeah Jun 12, 2024
a774ada
[Issue #40]: Setup pa11y-ci (#41)
rylew1 Jun 12, 2024
65e6840
[Issue #88] Adjust docker commands based on recent updates (#89)
chouinar Jun 17, 2024
c0acaae
[Issue #73]: Dynamic sitemap for pa11y-ci (#83)
rylew1 Jun 18, 2024
4c877fa
[Issue #96]: Opportunity listing page (first pass) (#97)
rylew1 Jun 20, 2024
4c1f78e
[Issue #74]: Pa11y API setup (#99)
btabaska Jun 21, 2024
4ed48da
[Issue #14] Setup utils for creating requests and parsing responses f…
chouinar Jun 25, 2024
80315b1
[Issue #24] Add GET /opportunity/:opportunityId/versions (#82)
chouinar Jun 25, 2024
94be4d0
Update python packages (#102)
chouinar Jun 25, 2024
817a1d3
Add make sprint-data-import and issue-data-import to import github sp…
aplybeah Jun 26, 2024
cbccbed
[Issue #16] Connect the API to use the search index (#63)
chouinar Jun 27, 2024
fd0b498
[Task]: Document the new steps for the analytics local database impor…
aplybeah Jul 1, 2024
46bbc1c
Add Logging to Sprint import (#130)
aplybeah Jul 1, 2024
78dfc83
[Issue #134] Renable static rendering (#85)
acouch Jul 5, 2024
9bd7bbf
Adjust transformation deletes to handle cascading deletes (#103)
chouinar Jul 8, 2024
79aa2cd
[Task] Update analytics db to use local.env and Remove Dynaconf from …
aplybeah Jul 26, 2024
85a16b7
Update API packages (#135)
chouinar Jul 29, 2024
d8dd736
[Issue #138] Setup s3 localstack (#161)
chouinar Jul 29, 2024
9150efd
[Issue #86] Download the search response as a CSV file (#87)
chouinar Jul 29, 2024
f89158d
[Issue #104] Refactor/restructure the transformation code (#112)
chouinar Aug 2, 2024
b4d0f53
[Task 163] Add post_date and close_date filters to search endpoint sc…
Rwolfe-Nava Aug 5, 2024
9eb8aa2
Add suspense boundary around search page results (#101)
acouch Aug 6, 2024
779de85
[Issue #167] Fix search page string translation (#169)
btabaska Aug 8, 2024
2e0666f
[Issue: #166] Create ecs task to export opportunity data as csv and j…
Rwolfe-Nava Aug 15, 2024
7245a9a
[Issue #170] Remove inline styling from pagination wrapper component …
btabaska Aug 23, 2024
de661aa
[Issue #174] Hide pagination component if no results are found (#175)
btabaska Aug 27, 2024
7f7103d
[Issue #181]Add no console log to eslint rules (#182)
btabaska Aug 28, 2024
2b23dd4
[Issue #184] Pin Python version to 3.12 + dependency updates (#185)
chouinar Aug 28, 2024
e78ebe6
[Issue #188] More filters in search schema (#189)
chouinar Sep 9, 2024
7fd6e25
Update API packages (#194)
chouinar Sep 9, 2024
0f175b1
[Issue #191] Cleanup makefile commands around OpenSearch (#192)
chouinar Sep 9, 2024
6b485e4
Update search query builder to support int and date range queries (#195)
chouinar Sep 9, 2024
b0df4e0
Specify docker platform in composer (#190)
acouch Sep 10, 2024
b0c93f2
[Issue #186] Onboarding Documentation Improvements (#193)
mdragon Sep 13, 2024
3a19603
[Issue #125] Setup agency tables for transformation (#127)
chouinar Sep 13, 2024
2854d43
[Issue #179] Incrementally load search data (#180)
chouinar Sep 13, 2024
4b0bd5b
[Issue #178] Finish connecting new search parameters to backend queri…
chouinar Sep 13, 2024
b5ff8c8
[Issue #106] Transform agency data (#157)
chouinar Sep 16, 2024
5f919a8
Update layout.tsx to fix 404 not rendering (#199)
acouch Sep 17, 2024
f9a58ab
[Issue #183] Opportunity Page Design Implementation (#196)
btabaska Sep 17, 2024
cf04b38
Update README.md
acouch Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
27 changes: 0 additions & 27 deletions .github/ISSUE_TEMPLATE/internal_30k.md

This file was deleted.

56 changes: 0 additions & 56 deletions .github/ISSUE_TEMPLATE/internal_adr.yml

This file was deleted.

1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/internal_task.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: Internal - Task
description: Describes an individual task that needs to be completed
title: "[Task]: "
labels: ["project: grants.gov"]
assignees:
- octocat
body:
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/ci-analytics.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,12 @@ jobs:
- name: Run tests
run: make test-audit

- name: Export GitHub data
run: make gh-data-export
# Both of these tasks are looking for github and slack auth
# - name: Export GitHub data
# run: make gh-data-export

- name: Run reports
run: make sprint-reports
# - name: Run reports
# run: make sprint-reports

vulnerability-scans:
name: Run Analytics Vulnerability Scans
Expand Down
73 changes: 73 additions & 0 deletions .github/workflows/ci-frontend-a11y.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: pa11y tests

on:
pull_request:
paths:
- frontend/**
- .github/workflows/ci-frontend-a11y.yml

jobs:
build:
name: Pa11y-ci tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./frontend

env:
NODE_VERSION: 20
LOCKFILE_PATH: ./frontend/package-lock.json
PACKAGE_MANAGER: npm

steps:
- name: Checkout source
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache-dependency-path: ${{ env.LOCKFILE_PATH }}
cache: ${{ env.PACKAGE_MANAGER }}

- name: Install dependencies
run: npm ci

- name: Create screenshots directory
run: mkdir -p screenshots-output

- name: Build project
run: npm run build

- name: Start server and log output
run: npm run start &

- name: Start API Server for search results
run: |
cd ../api
make init db-seed-local start &
cd ../frontend
# ensure the API wait script is executable
chmod +x ../api/bin/wait-for-api.sh
../api/bin/wait-for-api.sh
shell: bash

- name: Wait for frontend to be ready
run: |
# Ensure the server wait script is executable
chmod +x ./bin/wait-for-frontend.sh
./bin/wait-for-frontend.sh

- name: Run pa11y-ci
run: |
set -e # Ensure the script fails if any command fails
npm run test:pa11y-desktop
npm run test:pa11y-mobile
echo "pa11y-ci tests finished."

- name: Upload screenshots to artifacts
if: always()
uses: actions/upload-artifact@v3
with:
name: screenshots
path: ./frontend/screenshots-output
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,6 @@ dmypy.json

# vscode
.vscode/settings.json

# vim
*.swp
8 changes: 4 additions & 4 deletions .grype.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ ignore:
- fix-state: not-fixed
- fix-state: wont-fix
- fix-state: unknown
# Golang vulnerability inside of a python docker image. It's basically impossible to find
# out where the impacted golang vulnerability is coming from. The python image in question
# does not even have golang installed, yet somehow still there's a golang vulnerability.
# We are ignoring the finding because it would take undue effort to track down the source.
# Golang vulnerabilities inside of a python docker image. Both originate from lower level packages within the GitHub CLI:
# https://github.com/cli/cli/blob/trunk/go.mod#L101
# https://github.com/cli/cli/blob/trunk/go.mod#L161
- vulnerability: GHSA-4v7x-pqxf-cx7m
- vulnerability: GHSA-v6v8-xj6m-xwqh
# https://github.com/anchore/grype/issues/1172
- vulnerability: GHSA-xqr8-7jwr-rhp7
- vulnerability: GHSA-7fh5-64p2-3v2j
Expand Down
65 changes: 2 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,64 +1,3 @@
# [Simpler.Grants.gov](https://simpler.grants.gov/)
# This is an archived fork of https://github.com/HHS/simpler-grants-gov

A modernization effort for [Grants.gov](https://grants.gov/)

## About the Project

We want Grants.gov to be an extremely simple, accessible, and easy-to-use tool for posting, finding, sharing, and applying for federal financial assistance. Our mission is to increase access to grants and improve the grants experience for everyone. We’re improving the way applicants search for and discover funding opportunities, making it easier to find and apply. For federal grantmaking agencies, we’re making it easier for their communities to find the funding they need.

Go to [Simpler.Grants.gov](https://simpler.grants.gov/) to learn about our transparent process and what we’re doing now, or explore our existing user research and the findings that are guiding our work.

See [goals.md](./documentation/goals.md) for more information about the vision and goals for the project.

## Core Team

The core team on the grants.gov project is a small group of content strategists, designers, developers, and product managers working for and with the Department of Health and Human Services, and other federal agencies, and community volunteers.

An up-to-date list of core team members can be found in [MAINTAINERS.md](./MAINTAINERS.md). At this time, the project is still building the core team and defining roles and responsibilities. We are eagerly seeking individuals who would like to join the community and help us define and fill these roles.

## Repository Structure

- [./.github](./.github) contains Github specific settings files and testing, linting, and CI/CD workflows
- [./api](./api) contains an API built in Python using the Flask library
- [./bin](./bin) contains scripts for managing infrastructure
- [./documentation](./documentation) contains project guides, documentation, and decision records
- [./frontend](./frontend) contains a web application built using Next.js
- [./infra](./infra) contains Terraform modules and configuration for managing the AWS infrastructure

## Development

### API

Documentation for the API is linked to from the [API README.md](./api/README.md). For installation instructions, see the [development documentation](./documentation/api/development.md).

### Front-end

Documentation and development instructions for the front-end are provided in the [Front-end README.md](./frontend/README.md).

## Contributing

Thank you for considering contributing to an Open Source project of the US
Government! For more information about our contribution guidelines, see
[CONTRIBUTING.md](CONTRIBUTING.md) to learn more and join our community see our [wiki](https://wiki.simpler.hhs.gov).

## Security

For more information about our Security, Vulnerability, and Responsible
Disclosure Policies, see [SECURITY.md](SECURITY.md).

## Authors and Maintainers

For more information about our Authors and maintainers, see [MAINTAINERS.md](MAINTAINERS.md).

A full list of [contributors](https://github.com/HHS/simpler-grants-gov/graphs/contributors) can be found on GitHub.

## Public domain

This project is licensed within in the public domain within the United States,
and copyright and related rights in the work worldwide are waived through the
[CC0 1.0 Universal public domain
dedication](https://creativecommons.org/publicdomain/zero/1.0/).

All contributions to this project will be released under the CC0 dedication. By
submitting a pull request or issue, you are agreeing to comply with this waiver
of copyright interest.
Do not commit or do work here! Go to https://github.com/HHS/simpler-grants-gov for project updates.
2 changes: 0 additions & 2 deletions analytics/.gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
data

# Ignore dynaconf secret files
.secrets.*
7 changes: 7 additions & 0 deletions analytics/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,13 @@ sprint-data-export:
--project $(SPRINT_PROJECT) \
--output-file $(SPRINT_FILE)

gh-db-data-import:
@echo "=> Importing sprint data to the database"
@echo "====================================================="
$(POETRY) analytics import db_import \
--sprint-file $(SPRINT_FILE) \
--issue-file $(ISSUE_FILE)

roadmap-data-export:
@echo "=> Exporting project data from the product roadmap"
@echo "====================================================="
Expand Down
47 changes: 19 additions & 28 deletions analytics/config.py
Original file line number Diff line number Diff line change
@@ -1,32 +1,23 @@
"""Loads configuration variables from settings files and settings files
"""Loads configuration variables from settings files

Dynaconf provides a few valuable features for configuration management:
- Load variables from env vars and files with predictable overrides
- Validate the existence and format of required configs
- Connect with secrets managers like HashiCorp's Vault server
- Load different configs based on environment (e.g. DEV, PROD, STAGING)

For more information visit: https://www.dynaconf.com/
"""
from dynaconf import Dynaconf, Validator, ValidationError
import os
from typing import Optional
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

# reads environment variables from .env files defaulting to "local.env"
class PydanticBaseEnvConfig(BaseSettings):
model_config = SettingsConfigDict(env_file="%s.env" % os.getenv("ENVIRONMENT", "local"), extra="ignore") # set extra to ignore so that it ignores variables irrelevant to the database config (e.g. metabase settings)

settings = Dynaconf(
# set env vars with `export ANALYTICS_FOO=bar`
envvar_prefix="ANALYTICS",
# looks for config vars in the following files
# with vars in .secrets.toml overriding vars in settings.toml
settings_files=["settings.toml", ".secrets.toml"],
# add validators for our required config vars
validators=[
Validator("SLACK_BOT_TOKEN", must_exist=True),
Validator("REPORTING_CHANNEL_ID", must_exist=True),
],
)
class DBSettings(PydanticBaseEnvConfig):
db_host: str = Field(alias="DB_HOST")
port: int = Field(5432,alias="DB_PORT")
user: str = Field (alias="DB_USER")
password: str = Field(alias="DB_PASSWORD")
ssl_mode: str = Field(alias="DB_SSL_MODE")
slack_bot_token: str = Field(alias="ANALYTICS_SLACK_BOT_TOKEN")
reporting_channel_id: str = Field(alias="ANALYTICS_REPORTING_CHANNEL_ID")

# raises after all possible errors are evaluated
try:
settings.validators.validate_all()
except ValidationError as error:
list_of_all_errors = error.details
print(list_of_all_errors)
raise
def get_db_settings() -> DBSettings:
return DBSettings()
10 changes: 10 additions & 0 deletions analytics/local.env
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,13 @@ MB_DB_PORT=5432
MB_DB_USER=app
MB_DB_PASS=secret123
MB_DB_HOST=grants-analytics-db

###########################
# Slack Configuration #
###########################
# Do not add these values to this file
# to avoid mistakenly committing them.
# Set these in your shell
# by doing `export ANALYTICS_REPORTING_CHANNEL_ID=whatever`
ANALYTICS_REPORTING_CHANNEL_ID=DO_NOT_SET_HERE
ANALYTICS_SLACK_BOT_TOKEN=DO_NOT_SET_HERE
Loading
Loading