Add Car Assistant Natural Language example

.devcontainer/devcontainer.json

@@ -14,7 +14,8 @@
         },
         "ghcr.io/devcontainers-contrib/features/hatch:2": {
             "version": "latest"
-        }
+        },
+        "ghcr.io/devcontainers/features/docker-outside-of-docker": {}
     },
 
     // Use 'forwardPorts' to make a list of ports inside the container available locally.

typescript/examples/aicar-zod/package.json

@@ -0,0 +1,38 @@
+{
+  "name": "aicar-zod",
+  "version": "0.0.1",
+  "main": "dist/main.js",
+  "scripts": {
+    "build": "tsc -p src",
+    "postbuild": "copyfiles -u 1 src/**/*Schema.ts src/**/*.txt src/**/*.html dist"
+  },
+  "exports": {
+    "./*": [
+      "./dist/*.js"
+    ]
+  },
+  "author": "",
+  "license": "MIT",
+  "description": "",
+  "dependencies": {
+    "@bufbuild/buf": "^1.39.0",
+    "@bufbuild/protobuf": "^1.10.0",
+    "@bufbuild/protoc-gen-es": "^1.10.0",
+    "@connectrpc/connect": "^1.4.0",
+    "@connectrpc/connect-node": "^1.4.0",
+    "@connectrpc/protoc-gen-connect-es": "^1.4.0",
+    "assert": "^2.1.0",
+    "commander": "^12.1.0",
+    "dotenv": "^16.4.5",
+    "find-config": "^1.0.0",
+    "ts-progress": "^0.1.9",
+    "typescript": "^5.5.4",
+    "zod": "^3.23.8"
+  },
+  "devDependencies": {
+    "@types/assert": "^1.5.10",
+    "@types/find-config": "^1.0.4",
+    "@types/node": "^22.5.1",
+    "copyfiles": "^2.4.1"
+  }
+}

typescript/examples/aicar-zod/readme.md

@@ -0,0 +1,83 @@
+# aicar-zod
+---
+The goal of this example repo is to experiment with foundational AI tech (TypeChat) & models (GPT4O mini & PHI3 were tested) applied to in-car assistant scenarios.  The experiment approach is to utilize the real next generation of in-car standards (Kuksa, Covesa), augment those standards & frameworks with NL understanding tech powered by LLMs, to prove that the user experience can be improved with those technologies. 
+
+[TypeChat](https://microsoft.github.io/TypeChat/) - provides a schema first framework to improve integration between NL understanding and LLM facilitated function calling
+
+[Covesa VSS](https://covesa.global/) -
+Covesa Vehicle Signal Specification provides a standard schema for vehicle communications. The [COVESA Vehicle Signal Specification](https://covesa.github.io/vehicle_signal_specification/) (VSS) defines the names and semantics of a large set of _data entries_ that represent the current and/or intended state of a vehicle's sensors and actuators organized in a tree-like structure. For example, the vehicle's current speed is represented by the `Vehicle.Speed` entry.
+This standard has broad interest & support from key auto manufacturers and OEMs (BMW, Volvo, Bosch, Land Rover, etc.)
+
+[Kuksa](https://projects.eclipse.org/projects/automotive.kuksa) -
+Kuksa is a top level eclipse foundation project to define a software defined vehicle ecosystem based on a vehicle schema.  The VSS does not define how these signals are to be collected and managed within a vehicle, nor does it prescribe how other components in the vehicle can read or write signal values from and to the tree.  
+
+Kuksa provides a standard databroker & GRPC interface to enable communication with vehicles.
+
+**Kuksa Databroker** is a resource efficient implementation of the VSS signal tree and is intended to be run within a vehicle on a microprocessor based platform. It allows applications in the vehicle to interact with the vehicle's sensors and actuators using a uniform, high level gRPC API for querying signals, updating current and target values of sensors and actuators and getting notified about changes to signals of interest.
+
+<!-- black box diagram -- inputs/outputs -->
+
+```mermaid
+flowchart LR
+    A[AICar - TypeChat] --VSS--- DB
+    DB[Kuksa Databroker] --VSS--- P
+    P[Kuksa provider] --CAN frames etc---E
+    E[ECU] --- Sensor
+    E --- Actuator
+    style DB fill:#999999,stroke:#444444,color:#ffffff
+```
+
+At the right end, Kuksa providers implement the link between the Databroker and a vehicle's Electronic Control Units (ECU) to which the hardware sensors and actuators are physically attached.
+
+Data is usually exchanged with ECUs by means of a CAN bus or Ethernet based protocols like SOME/IP. Providers translate between the low level messages used by these protocols and the Databroker's high level gRPC API calls to update a sensor's current reading or to forward a set-point value to an actuator via its controlling ECU.
+
+
+### Features
+
+- 100% Open Source (Apache 2.0 license)
+- Written in Rust with an easy-to-use language agnostic gRPC interface
+- Lightweight (<4 MB statically compiled), allowing it to run on even small vehicle computers  
+
+
+
+## Setup and Environment Simulation 
+---
+### Databroker & Covesa
+The simplest way to get an example Databroker and VSS schema for experimentation is to use the official Databroker container.  
+
+> :memo: **Note:** The examples in this section do not use TLS nor access control. Please refer to the [Databroker User Guide](./doc/user_guide.md) for more sophisticated usage examples.
+
+### Prerequisites
+
+- [Docker Engine](https://docs.docker.com/engine/install/) or [Podman](https://podman.io/docs/installation)
+
+### Starting Databroker
+
+1. Start Databroker in a container attached to the host network on port 55556:
+
+   ```sh
+   docker run --rm -it -p 55556:55555 ghcr.io/eclipse-kuksa/kuksa-databroker:main --insecure --enable-databroker-v1
+   ```
+
+   > :bulb: **Tip:** You can stop the container using `ctrl-c`.
+
+### AICar MVE
+To test the sample in interactive mode, use the command line below.  This will leverage TypeChat to translate the natural language entered, validate it into the VSS scheme, and execute the GRPC call against the running databroker.
+   ```sh
+   node ./dist/main.js
+   ```
+
+### Android Auto Example App (not required)
+The Kuksa Companion android application provides a visual interface to see the actions taken by the AICar assistant.  It is not required, but helps illustrate how the full system interacts with the databroker & the resulting car.
+1.  Install Android Studio
+2.  Start the Pixel android emulator
+3.  Add the Kuksa Companion APK to the emulator
+4.  Start the emulator & connect to the databroker
+
+## Initial Test Results
+---
+
+
+
+## Next Steps & Investigations
+---

typescript/examples/aicar-zod/src/SDVCarActionZodSchema.ts

@@ -0,0 +1,199 @@
+import { z } from "zod";
+//import { VAL} from "./gen/kuksa/val/v1/val_connect";
+//import path from "path";
+//import { EntryRequest } from "./gen/kuksa/val/v1/val_pb";
+
+// """ Here is a list of signals which are used inside the app:
+// Door Control: bool
+// Vehicle.Cabin.Door.Row1.DriverSide.IsLocked bool
+// Vehicle.Cabin.Door.Row1.PassengerSide.IsLocked
+// Vehicle.Cabin.Door.Row2.DriverSide.IsLocked
+// Vehicle.Cabin.Door.Row2.PassengerSide.IsLocked
+// Vehicle.Body.Trunk.Rear.IsLocked
+// Vehicle.Cabin.Door.Row1.DriverSide.IsOpen
+// Vehicle.Cabin.Door.Row1.PassengerSide.IsOpen
+// Vehicle.Cabin.Door.Row2.DriverSide.IsOpen
+// Vehicle.Cabin.Door.Row2.PassengerSide.IsOpen
+// Vehicle.Body.Trunk.Rear.IsOpen
+// Temperature Control: int32
+// Vehicle.Cabin.HVAC.Station.Row1.Driver.Temperature
+// Vehicle.Cabin.HVAC.Station.Row1.Passenger.Temperature
+// Vehicle.Cabin.HVAC.Station.Row2.Driver.Temperature
+// Vehicle.Cabin.HVAC.Station.Row2.Passenger.Temperature
+// Light Control: bool
+// Vehicle.Body.Lights.Beam.High.IsOn
+// Vehicle.Body.Lights.Beam.Low.IsOn
+// Vehicle.Body.Lights.DirectionIndicator.Left.IsSignaling
+// Vehicle.Body.Lights.DirectionIndicator.Right.IsSignaling
+// Vehicle.Body.Lights.Fog.Front.IsOn
+// Vehicle.Body.Lights.Fog.Rear.IsOn
+// Vehicle.Body.Lights.Hazard.IsSignaling
+// Vehicle.Body.Lights.Parking.IsOn
+// Vehicle.Body.Lights.Running.IsOn
+// Tire Pressure:uint32
+// Vehicle.Chassis.Axle.Row1.Wheel.Left.Tire.Pressure
+// Vehicle.Chassis.Axle.Row1.Wheel.Right.Tire.Pressure
+// Vehicle.Chassis.Axle.Row2.Wheel.Left.Tire.Pressure
+// Vehicle.Chassis.Axle.Row2.Wheel.Right.Tire.Pressure """
+
+export const UnknownText = z.object({
+    type: z.literal('unknown'),
+    command: z.string().describe("The text that wasn't understood")
+});
+
+export const DataEntry = z.object({
+    //path: z.enum(['Vehicle.Cabin.HVAC.Station.Row1.Driver.Temperature', 'Vehicle.Cabin.HVAC.Station.Row2.Driver.Temperature', 'Vehicle.Cabin.HVAC.Station.Row1.Passenger.Temperature', 'Vehicle.Cabin.HVAC.Station.Row2.Passenger.Temperature']).describe("The path to the datapoint"),
+    path: z.string().describe("The path to the datapoint"),
+    value: z.string()
+});
+
+//export const GetTemperature = z.instanceof(DataEntry)
+export const GetTemperature = z.object({
+    type: z.literal('GetTemperature'),
+    command: z.object({entries: z.object({path: z.enum(['Vehicle.Cabin.HVAC.Station.Row1.Driver.Temperature', 
+        'Vehicle.Cabin.HVAC.Station.Row2.Driver.Temperature', 
+        'Vehicle.Cabin.HVAC.Station.Row1.Passenger.Temperature', 
+        'Vehicle.Cabin.HVAC.Station.Row2.Passenger.Temperature'])}).array()}),
+});
+
+export const GetDoorLockState = z.object({
+    type: z.literal('GetDoorLockState'),
+    command: z.object({entries: z.object({path: z.enum(['Vehicle.Cabin.Door.Row1.DriverSide.IsLocked', 
+        'Vehicle.Cabin.Door.Row2.DriverSide.IsLocked', 
+        'Vehicle.Cabin.Door.Row1.PassengerSide.IsLocked', 
+        'Vehicle.Cabin.Door.Row2.PassengerSide.IsLocked',
+        'Vehicle.Body.Trunk.Rear.IsLocked'])}).array()}),
+});
+export const SetTemperature = z.object({
+    type: z.literal('SetTemperature'),
+    command: z.object({updates: z.array(
+        z.object({
+          entry: z.object({
+            path: z.enum(['Vehicle.Cabin.HVAC.Station.Row1.Driver.Temperature', 
+              'Vehicle.Cabin.HVAC.Station.Row2.Driver.Temperature', 
+              'Vehicle.Cabin.HVAC.Station.Row1.Passenger.Temperature', 
+              'Vehicle.Cabin.HVAC.Station.Row2.Passenger.Temperature']),
+            value: z.object({ int32: z.number() })
+          }),
+          fields: z.string().array().nonempty().default(['FIELD_VALUE'])
+        })
+      )}),
+  });
+
+export const SetDoorLockState = z.object({
+    type: z.literal('SetDoorLockState'),
+    command: z.object({updates: z.array(
+        z.object({
+          entry: z.object({
+            path: z.enum(['Vehicle.Cabin.Door.Row1.DriverSide.IsLocked', 
+           'Vehicle.Cabin.Door.Row2.DriverSide.IsLocked', 
+           'Vehicle.Cabin.Door.Row1.PassengerSide.IsLocked', 
+           'Vehicle.Cabin.Door.Row2.PassengerSide.IsLocked','Vehicle.Body.Trunk.Rear.IsLocked']).describe("which door to lock"),
+            value: z.object({ bool: z.boolean() })
+          }),
+          fields: z.string().array().nonempty().default(['FIELD_VALUE'])
+        })
+      )}),
+  });
+  export const GetDoorOpenState = z.object({
+    type: z.literal('GetDoorOpenState'),
+    command: z.object({entries: z.object({path: z.enum(['Vehicle.Cabin.Door.Row1.DriverSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row2.DriverSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row1.PassengerSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row2.PassengerSide.IsOpen','Vehicle.Body.Trunk.Rear.IsOpen'])}).array()}),
+});
+
+  export const SetDoorOpenState = z.object({
+    type: z.literal('SetDoorOpenState'),
+    command: z.object({updates: z.array(
+        z.object({
+          entry: z.object({
+            path: z.enum(['Vehicle.Cabin.Door.Row1.DriverSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row2.DriverSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row1.PassengerSide.IsOpen', 
+           'Vehicle.Cabin.Door.Row2.PassengerSide.IsOpen','Vehicle.Body.Trunk.Rear.IsOpen']).describe("which door to open or close"),
+            value: z.object({ bool: z.boolean() })
+          }),
+          fields: z.string().array().nonempty().default(['FIELD_VALUE'])
+        })
+      )}),
+  });
+  //use to turn the lights off and on
+  export const SetLightState = z.object({
+    type: z.literal('SetLightState'),
+    command: z.object({updates: z.array(
+        z.object({
+          entry: z.object({
+            path: z.enum(['Vehicle.Body.Lights.Beam.High.IsOn', 
+                'Vehicle.Body.Lights.Beam.Low.IsOn', 
+                'Vehicle.Body.Lights.DirectionIndicator.Left.IsSignaling', 
+                'Vehicle.Body.Lights.DirectionIndicator.Right.IsSignaling', 
+                'Vehicle.Body.Lights.Fog.Front.IsOn', 
+                'Vehicle.Body.Lights.Fog.Rear.IsOn', 
+                'Vehicle.Body.Lights.Hazard.IsSignaling', 
+                'Vehicle.Body.Lights.Parking.IsOn', 
+                'Vehicle.Body.Lights.Running.IsOn']),
+            value: z.object({ bool: z.boolean() })
+          }),
+          fields: z.string().array().nonempty().default(['FIELD_VALUE'])
+        })
+      )}),
+  });
+    //use to get the state of the lights
+    export const GetLightState = z.object({
+        type: z.literal('GetLightState'),
+        command: z.object({entries: z.object({path: z.enum(['Vehicle.Body.Lights.Beam.High.IsOn', 
+            'Vehicle.Body.Lights.Beam.Low.IsOn', 
+            'Vehicle.Body.Lights.DirectionIndicator.Left.IsSignaling', 
+            'Vehicle.Body.Lights.DirectionIndicator.Right.IsSignaling', 
+            'Vehicle.Body.Lights.Fog.Front.IsOn', 
+            'Vehicle.Body.Lights.Fog.Rear.IsOn', 
+            'Vehicle.Body.Lights.Hazard.IsSignaling', 
+            'Vehicle.Body.Lights.Parking.IsOn', 
+            'Vehicle.Body.Lights.Running.IsOn'])}).array()}),
+    });
+    //use to get the pressure of the tires
+    export const GetTirePressure = z.object({
+        type: z.literal('GetTirePressure'),
+        command: z.object({entries: z.object({path: z.enum(['Vehicle.Chassis.Axle.Row1.Wheel.Left.Tire.Pressure', 
+            'Vehicle.Chassis.Axle.Row1.Wheel.Right.Tire.Pressure', 
+            'Vehicle.Chassis.Axle.Row2.Wheel.Left.Tire.Pressure', 
+            'Vehicle.Chassis.Axle.Row2.Wheel.Right.Tire.Pressure'])}).array().describe('must be an array of entries')}),
+    });
+    //use to set the pressure of the vehicle tires
+    export const SetTirePressure = z.object({
+      type: z.literal('SetTirePressure'),
+      command: z.object({updates: z.array(
+          z.object({
+            entry: z.object({
+              path: z.enum(['Vehicle.Chassis.Axle.Row1.Wheel.Left.Tire.Pressure', 
+                'Vehicle.Chassis.Axle.Row1.Wheel.Right.Tire.Pressure', 
+                'Vehicle.Chassis.Axle.Row2.Wheel.Left.Tire.Pressure', 
+                'Vehicle.Chassis.Axle.Row2.Wheel.Right.Tire.Pressure']),
+              value: z.object({ uint32: z.number() })
+            }),
+            fields: z.string().array().nonempty().default(['FIELD_VALUE'])
+          })
+        )}),
+    });
+
+export const SDVCarActions = z.object({
+    actions: z.discriminatedUnion("type", [
+        GetDoorLockState.describe('Use this to get the state of the door locks'), 
+        SetTemperature.describe('Use this to set the temperature'), 
+        GetTemperature.describe('Use this to get the temperature'), 
+        GetLightState.describe('Use this to get the state of the lights'), 
+        SetLightState.describe('Use this to turn the lights off and on'),
+        SetDoorLockState.describe('Use this to lock and unlock the doors.  use GetDoorOpenState for opening and closing doors'), 
+        GetTirePressure.describe('Use this to get the tire pressure'),   
+        GetDoorOpenState.describe('Get the door open state'), 
+        SetDoorOpenState.describe('Use this to open and close the doors.  use SetDoorLockState for locking and unlocking doors'),
+        SetTirePressure.describe('Use this to set the tire pressure'), 
+        UnknownText]).array()
+});
+
+export const SDVCarSchema ={
+    SDVCarActions: SDVCarActions.describe("The actions to perform on the car"),
+    UnknownText: UnknownText.describe("Use this type for order items that match nothing else")
+};
+