Skip to content

Welcome to CarbonORM 👋

View all our documentation at www.CarbonORM.dev!

The CarbonORMs are sometimes called C6 since Carbon is the sixth element in the periodic table of the elements. We aim to enable developers to query SQL in their middleware and frontend in a safe, constant, and syntactically pleasing way. Our Rest-based ORM auto-generates 1-1 type references to your tables and columns in JavaScript, TypeScript, and your middleware language (PHP, JAVA, PYTHON, C#, ...). We are still working to support all major languages, and help is very much welcome! Payloads between the frontend and middleware are JSON based, and working with SQL in your middleware will use objects and array language-specific datatypes. Our generated C6 typings ensure a few things:

  1. Large organizations with a frontend and backend team separated will not see the timing issues typical projects face. Generally, most operations can be done using a single table, and all connections are already generated to interface from the front end; front-end teams will not have to wait for the backend to create the necessary routes required for new features. This leaves backend teams focusing on validations rather than boilerplate and the frontend without bottlenecks.
  2. Type definitions allow your editors to make better coding suggestions.
  3. If you change a column name, every internal query in your project middleware that uses the column constant can be found and changed. Depending on the middleware, accessing the old value would cause a compile-time or invalid reference error. This reduces the load on your SQL servers and in many cases will prevent publishing invaild code all togeather.
  4. Invalid external requests, think bad-actor, that use invalid column references raise an Error in the middleware, not the backend SQL server reducing load and possible Distributed Denial-of-Service (DDOS) abuse potentially affecting every user.
  5. Teams of developers can ensure that the SQL data schema stays up-to-date with each other. We have identified general change rules that allow CarbonORM to update projects on different servers or databases automagically. Some examples include auto-creating columns, primary and foreign key references, and changing column types. CarbonORM has also standardized where making advanced table updates should be placed. When changes exist that cannot be deterministically automated by us, an error will be thrown!

Language Support

This concept began as a simple PHP project and have slowly began integrating with NodeJs and the React Frontend runtime. Other language repositories are in progress but no other are ready to be use in a production setting. Focus for future development has been restricted to minor and patch releases in CarbonPHP. CarbonNode will gain many of the feature CarbonPHP already offer. Development in CarbonC will begin and take precidence to help introduce module support for many laguages with one package. For example Ruby, Python, Node, and PHP are all written in C so writing CarbonORM in C is an enevitabillity that will increase preformance for all of these higher level languages

Removing the risk of SQL injection entirely with prepared statements

Most languages support prepared statements, but the boilerplate code becomes repetitive in large projects. We eliminate the need for repetition by looping through these data structures and validating against our auto-generated code data definitions.

How do we stop arbitrary SQL execution? (frontend testing is now necessary)

In local environments, we don't ;) This allows front-end engineers to move quickly. In our CI/CD pipelines, the frontend tests will quietly store all prepared statements it queried during the test. Our ORM conveniently provides this information with every request. After every test completes, C6 will compile these 'Valid SQL procedures' into a single object that the middleware will use for its validation in non-local environments. Any SQL executed from the frontend not existing in our valid SQL array will be logged and rejected.

Autogenerated with persistence!

While editing generated code is typically insufficient, we look to change this common practice. We believe there is no better place to put restful helper methods than the table class they operate on. We use custom complex regexes to capture these custom methods and persist them from revision to revision. We are pleased about how seamlessly this works and improves the DevOps experience.

Finegrane control using hooks

It's possible to process complex Transactional code, kill a request based on the SQL generated, validate a column and its arguments each time it is processed, preprocess the request before the rest code is run, run global validations on every column encountered, rollback a transaction based on the results of a query, and even more possibilities.

Transactions and Table Locking

Queries can be locked using MySQL InnoDB ^8 FOR UPDATE and FOR SHARE, which we can be used by adding self::LOCK => self::FOR_UPDATE, and self::LOCK => self::FOR_SHARE to the query body.

Frontend -> Backend JSON Syntax

As C6-enabled SQL queries are just objects and arrays, queries can be stored and abstracted in variables or constants and modified without string manipulation. Hooray! Middleware languages have varying syntax for representing these datatypes. To view language-specific examples for middleware queries, please check out your respective middleware repository:

We will use the standard frontend JavaScript (TypeScript) syntax for our examples below. GET requests are JSON based and serialized into query parameters using qs.

Qs.stringify(params, {arrayFormat: 'indices', indices: true, skipNulls: false, strictNullHandling: true})

QS is a dependency of CarbonNode and thereby CarbonReact. It will automatically be added to your package when you install either.

"@types/qs": "^6.9.8",
"qs": "^6.11.1",

Apache Setup

httpd and apache2 are the same application - just that some Linux distributions refer to it differently within package managers and config files. RedHat-based distros (CentOS, Fedora) refer to it as httpd while Debian-based distros (Ubuntu) refer to it as apache. The commands below use httpd, but if that fails you should try replacing it with apache2

  1. Open Apache configuration file. ex /usr/local/etc/httpd/httpd.conf
    • If you do not know where it is you can run httpd -t -D DUMP_INCLUDES to output the locations. In rare cases, if multiple versions or instances of apache exist on the machine the httpd executable may not output the correct information. Using where httpd can potentially show multiple executables you may explicitly try /usr/local/bin/httpd -t -D DUMP_INCLUDES
  2. Edit or add the file to include LimitRequestLine 40000 and LimitRequestFieldSize 40000
  3. Restart apache httpd restart or apache2 restart

The limit of 40,000 is arbitrary and can be adjusted. We recommend you use a this value or even 20,000 to help catch progromatic error you may cause. A common issues occurs when using MySQL IN Clause, which can have its own limits, to filter out data already slected and cached in the ui/frontend. As example, passing ten thousand ID's is not a good solution and we might want that to actually fail so we can write a better MySQL join and paginate properly. The C6 Frontend bindings have built in pagination support. Checkout all our documentation out at CarbonORM.dev.

Nginx Setup

Locate NGINX Configuration File: The main NGINX configuration file is typically located at /etc/nginx/nginx.conf. However, the specific file to edit can vary depending on your server setup. It might be in a site-specific configuration file in the /etc/nginx/sites-available/ directory.

Edit the Configuration File:

Open the configuration file in a text editor. For example, using nano:

sudo nano /etc/nginx/nginx.conf

Find the http block in the configuration file. Inside this block, add or modify the large_client_header_buffers directive. For a 20,000 character URL limit, you need to set the buffer size larger than 20,000. The directive takes two parameters: the number of buffers and the size of each buffer. For example:

http {
    ...
    large_client_header_buffers 4 24k;
    ...
}

Here, 4 is the number of buffers, and 24k is the size of each buffer. This will roughly equal 20,000 character limit. To move to 80,000 change the directive to large_client_header_buffers 4 100k;. Adjust the size as needed to accommodate your URL length requirement. Test the Configuration: After editing, it's important to test the configuration for syntax errors:

sudo nginx -t

Reload NGINX: If the configuration test is successful, reload NGINX to apply the changes:

sudo systemctl reload nginx

Verify Changes: Test with a long URL to ensure the new configuration is working as expected.

You can refer to this StackOverflow post.

Browser Support

Our middleware ORM's do not require a browser. The CarbonReact and Javascript bindings send full JSON SQL with does require large request URI's. This is typically not an issue for most applications, but may require a few custom configurations to your CDN and Apache/Nginx/etc server. This also means, for now, we genearlly do not support Microsoft Edge browser. A plan to use WebSockets to optionally facillitate requests would allow Edge support, but no timeline has been set. Many small simple select statments will work. When a Select gets complex with joins you generally will see anywhere from 2,000 - 12,000 characters being sent to the backend system. Queries may be even longer and C6 defines no limit on its own.

Browser Max URL Length Notes
Microsoft IE 2,048 The Internet Explorer 11 desktop application went out of support for certain operating systems starting June 15, 2022
Microsoft Edge 4,741 "... Jerry Han's answer was of 4035 consistent with my observations in June, 2020. However, I've observed that the correct answer depends on the version of Edge. As of today, however, I'm seeing a 4741 character test link (which failed last summer), work just fine in the latest version (89.0.744.63) of Edge."
Firefox 65,536 As of June 2022, Firefox allows URLs to be unlimited in length, but the location bar will not display URLs after 65,536 characters.
Chrome 2,097,152
Safari 80,000
Opera unlimited

CDNs also impose limits on URI length, and will return a 414 Too long request when these limits are reached. Generally, C6 Rest based requests will/should not be cached on a CDN level and should provide no indexing to search engines. This is a general precident for all web based API's. Your applications rest endpoint defaults to /rest/ but may be customized with a prefix. This endpoint should be blocked from caching.

CDN Limit
Fastly 8,096
CloudFront 8,096
CloudFlare 32,384

GET (SELECT)

SIMPLE

Here is an open-source example from my time at Drop-In Gaming. Though full examples are available at the associated links, the following are stripped of imports. getGameList.tsx

export default restApi<{}, iDig_Chat_Global_Messages, {}, iGetC6RestResponse<iDig_Chat_Global_Messages>>({
    axios: DropVariables.axios,
    tableName: C6.dig_chat_global_messages.TABLE_NAME,
    requestMethod: GET,
    queryCallback: {
        [C6.SELECT]: [
            dig_chat_global_messages.GLOBAL_MESSAGE_ID,
            dig_chat_global_messages.GLOBAL_MESSAGE_FROM_USER,
            dig_chat_global_messages.GLOBAL_MESSAGE,
            dig_chat_global_messages.GLOBAL_MESSAGE_DATETIME,
            dig_chat_global_messages.GLOBAL_MESSAGE_ADVERTISEMENT,
            dig_chat_advertisements.ADVERTISEMENT_ACTIVE,
            dig_chat_advertisements.LAST_SHOWN_AT,
            [um_metadata.UM_VALUE, C6.AS, 'profile_photo'],
            users.DISPLAY_NAME,
            users.USER_NICENAME
        ],
        [C6.JOIN]: {
            [C6.INNER]: {
                [users.TABLE_NAME]: {
                    [dig_chat_global_messages.GLOBAL_MESSAGE_FROM_USER]: users.ID
                }
            },
            [C6.LEFT_OUTER]: {
                [um_metadata.TABLE_NAME]: {
                    [um_metadata.USER_ID]: users.ID,
                    [um_metadata.UM_KEY]: 'profile_photo',
                },
                [dig_chat_advertisements.TABLE_NAME]: {
                    [dig_chat_advertisements.ADVERTISEMENT_ID]: dig_chat_global_messages.GLOBAL_MESSAGE_ADVERTISEMENT
                }
            }
        },
        [C6.GROUP_BY]: [
            dig_chat_global_messages.GLOBAL_MESSAGE_ID,
            um_metadata.UM_VALUE
        ],
        [C6.PAGINATION]: {
            [C6.ORDER]: {
                [dig_chat_global_messages.GLOBAL_MESSAGE_DATETIME]: C6.DESC
            },
            [C6.PAGE]: 1,
            [C6.LIMIT]: 100
        },
    },
    responseCallback: response => {

        const restData = response?.data?.rest A|| [];

        DigApi.digApi.setState((previousState) => ({
            globalMessages: [...previousState.globalMessages || [], ...restData],
        }));

    }
})

Its associated JEST test is even simpler. getGameList.test.tsx

describe('Get Games List', () => {

    test('Asset Game List Not Empty', async () => {

        let bootstrap: DigApi = DigApi.digApi;

        // @link https://reactjs.org/link/wrap-tests-with-act
        await act(async () => {

            /* fire events that update state */
            await getGameList();

            expect(bootstrap.state.backendThrowable).toEqual([]);

            expect(bootstrap.state.games).not.toBeNull();

            expect(bootstrap.state.games).not.toBeUndefined();

        });

        expect(bootstrap.state.games?.length).toBeGreaterThan(12)

    }, 31000);

});

DELETE

See the full example here.

deleteFeatureGroupReference.tsx

export default restApi<{
    featureId: number,
    groupId: number
}, iDig_Feature_Group_References, {}, iDeleteC6RestResponse>({
    axios: DropVariables.axios,
    tableName: C6.dig_feature_group_references.TABLE_NAME,
    requestMethod: DELETE,
    queryCallback: (request) => {
        request.success = 'Successfully deleted the feature from the group.'
        request.error = 'Failed to remove the feature from the group. Please try again later.'
        return {
            [C6.WHERE]: {
                [C6.dig_feature_group_references.FEATURE_ENTITY_ID]: request.featureId,
                [C6.dig_feature_group_references.GROUP_ENTITY_ID]: request.groupId,
            }
        }
    },
    responseCallback: (_response, request, id) => {

        id && updateRestfulObjectArray<iGroups>((previousState) => {

            const fullGroup: iGroups | undefined = previousState.groups?.find((group: iGroups) => group.entity_id === request.groupId);

            const fullFeature: iDig_Features | undefined = previousState.features?.find((feature: iDig_Features) => feature.feature_entity_id === request.featureId);

            const index : number|undefined = fullGroup?.feature_code?.indexOf(fullFeature?.feature_code ?? '');

            if (index && index > -1) { // only splice array when item is found
                fullGroup?.feature_code?.splice(index, 1); // 2nd parameter means remove one item only
            }

            return fullGroup ? [fullGroup] : [];

        }, "entity_id", "groups")

    }
});

PUT (UPDATE)

The code directly blow is executing a change using the requestMethod: PUT,.

await putUser({
    ID: myUser?.id,
    user_email: '[email protected]'
})

The full code blow can be found here. putUser.tsx

export default restApi<{ ID: number }, iUsers,{}, iPutC6RestResponse<iUsers>>({
    axios: DropVariables.axios,
    tableName: C6.users.TABLE_NAME,
    requestMethod: PUT,
    queryCallback: (request) => {
        request.success = 'Successfully updated user information!'
        request.error = 'An error occurred while updating user information!'
        return request
    },
    responseCallback: (response, request) => {

        console.log('response from', response)

        if (undefined !== request.user_pass) {

            return;

        }

        updateRestfulObjectArray<iUsers>([removeInvalidKeys(request, [users])], "ID", "users");

    }

});

POST (INSERT)

The registration example.

postUser({
    user_login: userSignUpAndInInformation.user_login,
    user_email: userSignUpAndInInformation.user_email,
    user_birthday: dateOfBirth,
    organization_id: organization_id,
    success: () => {
        return 'Account created!'  // this will appear in a toast notification
    }
}).then(axiosResponse => console.log(axiosResponse))
  .error(error => console.log(error))

The promise returned by the example above does not need to be awaited as the responseCallback in the code below is usually responcible for handing the initial parsing and placed into state.

The gerneral code below.

export default restApi<iPostUser, iUsers & iDig_Temp_Invite & iDig_Parent_User & iDig_User_Info & iDig_User_Organizations, {}, iPostC6RestResponse<iUsers>>({
    axios: DropVariables.axios,
    tableName: [C6.users.TABLE_NAME, C6.dig_temp_invite.TABLE_NAME, C6.dig_parent_user.TABLE_NAME, C6.dig_user_info.TABLE_NAME, C6.dig_user_organizations.TABLE_NAME, ],
    requestMethod: POST,
    queryCallback: (request) => {
        request.success ??= 'Successfully created your account! You may now log in using your new credentials.'
        request.error = 'An unknown issue occurred while creating your account.'
        return request
    },
    responseCallback: (_response, request, id) => {
        // ... responseCallback code here ...
    }

});

How does autogenerating the definitions work?

Remember that the business logic is shared; all that is needed is class (object) files with the necessary column data; usually presented as constants. We provide an array with a breakdown of types, foreign key references, and column-specific information. Our editable generations include these class variables: a map for column => regex validation, a complex map used for assigning callable functions to column/validation rest query hooks, and finally, a simple array of callables filled with complex SQL database update procedures.

  1. We generate a Data Definition Language (DDL) file using common tools like mysqldump.
  2. Parse this data using regex or string manipulation and output the data types for JavaScript/Typescript and the Middleware language.
  3. Profit.

You can view a generated table file (carbon_users) which includes custom user methods that will persist across regeneration at my open-source php starter template CarbonORM/CarbonReactBootstrap.

Pinned Loading

  1. CarbonPHP CarbonPHP Public

    CarbonPHP [C6] is a PHP Library ^8.2 ⊃ { MySQL ORM ∪ WebSockets ∪ Sessions }

    PHP 10 3

  2. CarbonORM.dev CarbonORM.dev Public template

    React Web Documentation

    TypeScript

  3. CarbonReact CarbonReact Public

    React bindings for Carbon ORM

    TypeScript

Repositories

Showing 10 of 16 repositories
  • CarbonPHP Public

    CarbonPHP [C6] is a PHP Library ^8.2 ⊃ { MySQL ORM ∪ WebSockets ∪ Sessions }

    CarbonORM/CarbonPHP’s past year of commit activity
    PHP 10 MIT 3 0 1 Updated Jul 19, 2024
  • CarbonNode Public

    Node JS ORM

    CarbonORM/CarbonNode’s past year of commit activity
    TypeScript 0 MIT 0 1 0 Updated Jun 1, 2024
  • CarbonC Public

    Writing CarbonORM in C allows us to port the library to languages like Ruby, Python, PHP, Node, and more as an internal module.

    CarbonORM/CarbonC’s past year of commit activity
    C 0 0 0 0 Updated May 31, 2024
  • .github Public

    Our Mission

    CarbonORM/.github’s past year of commit activity
    0 MIT 0 0 0 Updated May 31, 2024
  • CarbonReact Public

    React bindings for Carbon ORM

    CarbonORM/CarbonReact’s past year of commit activity
    TypeScript 0 MIT 0 0 0 Updated May 28, 2024
  • CarbonORM.dev Public template

    React Web Documentation

    CarbonORM/CarbonORM.dev’s past year of commit activity
    TypeScript 0 MIT 0 0 0 Updated Apr 28, 2024
  • CarbonWordPress Public

    Wordpress bindings and GUI for CarbonPHP. A wordpress Plugin.

    CarbonORM/CarbonWordPress’s past year of commit activity
    PHP 0 MIT 0 0 0 Updated Apr 12, 2024
  • CarbonWordPressExample Public template

    POC and testing ground for the CarbonWordpress plugin

    CarbonORM/CarbonWordPressExample’s past year of commit activity
    PHP 0 MIT 0 0 0 Updated Apr 4, 2024
  • CarbonGo Public

    The CarbonORM system implemented in Go

    CarbonORM/CarbonGo’s past year of commit activity
    0 MIT 0 0 0 Updated Feb 12, 2024
  • CarbonJava Public

    Run CarbonORM Queries in Java

    CarbonORM/CarbonJava’s past year of commit activity
    Java 0 MIT 0 0 0 Updated Jan 28, 2024

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…