-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[pre-commit.ci] auto fixes from pre-commit hooks
- Loading branch information
1 parent
d7fd0c3
commit f4a3a40
Showing
1 changed file
with
133 additions
and
128 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,131 +1,136 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## FHIR to Tables\n", | ||
| "\n", | ||
| "This notebook provides a tutorial on how to use functionality from the phdi library to achieve the following:\n", | ||
| "1. Read a user-defined schema config file that specifies one or more tables.\n", | ||
| "2. Query a FHIR server to extract the data specified in the schema.\n", | ||
| "3. Transform the data into a structured format.\n", | ||
| "4. Load the data into one of the supported persistent tabular file formats.\n", | ||
| "5. Print a summary of the tables that were created.\n", | ||
| "\n", | ||
| "### Getting Started\n", | ||
| "\n", | ||
| "To make it easier to get started we have included two additional files in this direcotry:\n", | ||
| "1. A simple example schema config file called `example_schema.yaml`.\n", | ||
| "2. A `requirements.txt` file that can be used to install the `phdi` library in your Python environment by running `pip install -r requirements.txt`.\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Import Dependencies\n", | ||
| "- We'll use pathlib to easily handle our file paths.\n", | ||
| "- Load functions and credential manager class from the phdi library to easily extract and load data from a FHIR server into tables." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from pathlib import Path\n", | ||
| "from phdi.tabulation import print_schema_summary\n", | ||
| "from phdi.fhir.tabulation import generate_all_tables_in_schema\n", | ||
| "from phdi.cloud.azure import AzureCredentialManager" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Set Required Parameters\n", | ||
| "##### Description of parameters:\n", | ||
| "- **schema_path**: The path to a schema config file. We have provided a simple example of a schema config file in this directory called \"example_schema.yaml\". Currently only YAML files are supported, but support for JSON will be added in the future.\n", | ||
| "- **output_path**: The path to the directory where the tables will be written. If this directory does not exist it will be created. We have set this by default to your current directory.\n", | ||
| "- **output_format**: The format of the files that the tables will be written to. Either parquet or csv is currently supported.\n", | ||
| "- **fhir_url**: The URL of the FHIR server you wish to extract data from.\n", | ||
| "- **cred_manager**: A credential manager for authentication with the FHIR server. \n", | ||
| " " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "schema_path = Path(\"example_schema.yaml\") # Path to a schema config file.\n", | ||
| "output_path = Path(\".\") # Path to directory where tables will be written\n", | ||
| "output_format = \"parquet\" # File format of tables\n", | ||
| "fhir_url = \"https://your_fhir_url\" # The URL for a FHIR server\n", | ||
| "cred_manager = AzureCredentialManager(fhir_url) # Credential manager for authentication with the FHIR server." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Make the Tables\n", | ||
| "Call the make_schema_tables function from phdi to generate tables according to the schema config file from data in the specified FHIR server." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "generate_all_tables_in_schema(schema_path, output_path, output_format, fhir_url, cred_manager)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Display Summary of the Tables\n", | ||
| "- Use the print_schema_summary function from phdi to get a brief summary of each table that was created." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "print_schema_summary(output_path, display_head=True)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3.9.12 64-bit ('phdi')", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3" | ||
| }, | ||
| "vscode": { | ||
| "interpreter": { | ||
| "hash": "e103a5e2cf332ada5e63cb8ed09836b0c0753cc78d7570640dbe1135af568ea1" | ||
| } | ||
| } | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## FHIR to Tables\n", | ||
| "\n", | ||
| "This notebook provides a tutorial on how to use functionality from the phdi library to achieve the following:\n", | ||
| "1. Read a user-defined schema config file that specifies one or more tables.\n", | ||
| "2. Query a FHIR server to extract the data specified in the schema.\n", | ||
| "3. Transform the data into a structured format.\n", | ||
| "4. Load the data into one of the supported persistent tabular file formats.\n", | ||
| "5. Print a summary of the tables that were created.\n", | ||
| "\n", | ||
| "### Getting Started\n", | ||
| "\n", | ||
| "To make it easier to get started we have included two additional files in this direcotry:\n", | ||
| "1. A simple example schema config file called `example_schema.yaml`.\n", | ||
| "2. A `requirements.txt` file that can be used to install the `phdi` library in your Python environment by running `pip install -r requirements.txt`.\n" | ||
| ] | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Import Dependencies\n", | ||
| "- We'll use pathlib to easily handle our file paths.\n", | ||
| "- Load functions and credential manager class from the phdi library to easily extract and load data from a FHIR server into tables." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from pathlib import Path\n", | ||
| "\n", | ||
| "from phdi.cloud.azure import AzureCredentialManager\n", | ||
| "from phdi.fhir.tabulation import generate_all_tables_in_schema\n", | ||
| "from phdi.tabulation import print_schema_summary" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Set Required Parameters\n", | ||
| "##### Description of parameters:\n", | ||
| "- **schema_path**: The path to a schema config file. We have provided a simple example of a schema config file in this directory called \"example_schema.yaml\". Currently only YAML files are supported, but support for JSON will be added in the future.\n", | ||
| "- **output_path**: The path to the directory where the tables will be written. If this directory does not exist it will be created. We have set this by default to your current directory.\n", | ||
| "- **output_format**: The format of the files that the tables will be written to. Either parquet or csv is currently supported.\n", | ||
| "- **fhir_url**: The URL of the FHIR server you wish to extract data from.\n", | ||
| "- **cred_manager**: A credential manager for authentication with the FHIR server. \n", | ||
| " " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "schema_path = Path(\"example_schema.yaml\") # Path to a schema config file.\n", | ||
| "output_path = Path(\".\") # Path to directory where tables will be written\n", | ||
| "output_format = \"parquet\" # File format of tables\n", | ||
| "fhir_url = \"https://your_fhir_url\" # The URL for a FHIR server\n", | ||
| "cred_manager = AzureCredentialManager(\n", | ||
| " fhir_url\n", | ||
| ") # Credential manager for authentication with the FHIR server." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Make the Tables\n", | ||
| "Call the make_schema_tables function from phdi to generate tables according to the schema config file from data in the specified FHIR server." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "generate_all_tables_in_schema(\n", | ||
| " schema_path, output_path, output_format, fhir_url, cred_manager\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Display Summary of the Tables\n", | ||
| "- Use the print_schema_summary function from phdi to get a brief summary of each table that was created." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "print_schema_summary(output_path, display_head=True)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3.9.12 64-bit ('phdi')", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3" | ||
| }, | ||
| "vscode": { | ||
| "interpreter": { | ||
| "hash": "e103a5e2cf332ada5e63cb8ed09836b0c0753cc78d7570640dbe1135af568ea1" | ||
| } | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |