Option to set up and keep resources (containers) after AppHost shutdown

[aL3891]

One issue we had with project tye was that containers where deleted when the host was stopped. For local development we create a local copy of our dev database but settings this up and downloading all the data takes a while so we dont want to do that on every startup. instead we ended up having some separate scripts to set up the containers that where ment to stick around

It would be great if
a) there was an option to not delete containers on apphost shut down, the host would check if the container was there, if not, create it and start it and then not touch it.

b) There was a way to execute some code when the container was created the first time and optionally when it exists but wasn't running. in our case then we'd put our script for downloading the dev database in the callback for when the database container was created

[dbreshears]

DCP side of this is completed and should be integrated soon, Changes in the app-model will still need to be made.

[mitchdenny]

@JamesNK

[DamianEdwards]

Do we have an API proposal for the Aspire.Hosting side of this? Something like:

var redis = builder.AddRedis("cache")
                   .WithLifetime(ContainerLifetime.Persistent);

Also, to be clear, is this just about the first request (lifetime) or has DCP added support for executing scripts on container start too?

[danegsta]

We've only implemented support for specifying a persistent container lifetime at this point.

Another option for the API might be:

var redis = builder.AddRedisContainer("cache")
                   .WithPersistence();

@karolz-ms and I were discussing and our current thinking is that fully managed (lifetime matches that of the Aspire AppHost) and persistent (created if it doesn't exist and left running) are the only two scenarios that really make sense for container resources. We initially thought about handling a third scenario (require the resource to already exist and never create it), but quickly realized that all you really would need to be able to configure in Aspire for that scenario would be a set of external service endpoints.

For the scenario of initializing a database when a container is first created, one pattern that would work right now would be to use a custom image rather than the default database image. Effectively, create a new custom image based on the database base image and override the entrypoint with a script that can both start the database and download the needed data, that way it would run whenever the container starts.

[DamianEdwards]

For the scenario of initializing a database when a container is first created, one pattern that would work right now would be to use a custom image rather than the default database image. Effectively, create a new custom image based on the database base image and override the entrypoint with a script that can both start the database and download the needed data, that way it would run whenever the container starts.

Agreed on this as a good approach in general if a container is desired. In scenario where not deploying the database as a container though, I think this is yet another question of how to initialize databases that I still think is out of scope of solving in Aspire directly.

[joperezr]

FYI @mitchdenny for thoughts around integration with AppModel

[mitchdenny]

I think that there are two connected issues here around Durability and Persistence. A lot of people who want durability possibly want it for the purposes of persistence (i.e. they don't care if the container gets shutdown but they do care whether the data gets destroyed).

So we made some progress around persistence as a concept for the Azure emulators. We now have the following API:

var builder = DistributedApplication.CreateBuilder(true);
var blobs = builder.AddAzureStorage("storage")
                   .UseEmulator(container => {
                     container.UsePersistence();
                   )
                   .AddBlobs();

The callback on UseEmulator(...) is new, as is the UsePersistence(...) extension method. I'm thinking that enabling persistence is probably going to be a per resource extension because based on what I learned from Azure Storage and Cosmos ... it usually involves setting some environment variables and extra options.

I think that durability will similarly be a per resource concept where on a per resource basis we need to decide whether a container needs to be restarted next time the AppHost is launched. I'm thinking some kind of configuration hash that is stored by DCP along with the executing container that we would recalculate and compare each time we F5 (if the user has opted that resource into being durable).

This would be super useful for resources like Cosmos DB where the emulator takes forever to spin up.

This would need some more design work and I think we probably are out of time for P4. This means it either becomes a post-GA thing, or we look at adding it as a feature later in the cycle.

[DamianEdwards]

I agree that persistence is the right priority for now (and likely v1). I like the idea of a common method name with resource-specific implementation that turns on persistence. Not sure about the name though? Perhaps it would be better to keep it closer to the implementing concept of a volumes and keep the With prefix, e.g. WithVolume() or WithDataVolume(), which naturally leads itself to supporting overloads to allow specifying the name, WithVolume("mydata"). That would enable sharing volumes between different projects, etc.

[davidfowl]

We're punting this out of the first version right? Moving this to backlog

[davidroth]

👍for configuring the liftime of containers.
With the current model, it is quite expensive to bootstrap all the external containers over and over again when I need to restart my application.

For example, booting pgadmin takes several seconds and uses a lot of CPU cycles for a few seconds.
Also, when I stop my application (aspire host), I typically want to be able to access pgadmin or similar services.
Typically our developers work on a project all day, so it makes sense to keep external services (postgres, redis, etc) running when the application is stopped.

Thats also how we worked with the docker-compose-dev.yml file aproach (start once in the morning, nevery stop :-) )

[mitchdenny]

Linking in #1623

[mitchdenny]

Linked issue #1623 adds in the idea for making containers persistent as well.

[rosieks]

It would be nice to keep all containers running after stopping app but also stop them if someone close solution/visual studio and load another project. This might optimize resource usage for those that work on multiple projects.

[mitchdenny]

@danegsta @karolz-ms moving this to 9.0. As per the conversations this week I think this definately makes the cut.

[AndiRudi]

I just implemented Aspire (with Azurite Emulator) and realised that all blobs are gone when the solution is stopped. This impacts the development quite a bit. The only workaround I see is to seed on every start, right? Or is there a workaround I did not see?

[mitchdenny]

I'm mobile so I can't validate this but pseudo code to address this problem is:

builder.AddAzureStorage("storage").RunAsEmulator(c => {
   c.WithDataBindMount(...);
});

When spinning up Azurite this will mount its data into a local folder so that when you restart the app host the data will still be there.

[davidfowl]

We're enabling this for containers in Aspire 9

[joperezr]

Updated title to only have containers as that's the focus for 9.0.

[joperezr]

@onionhammer sorry I don't think I'm following. Can you point to the documentation that suggests differently? I'm also not sure how that relates to EF. This issue was tracking the work to keep resources (both containers and executables) alive after an AppHost shutdown. We are scoping the work we plan to get to by 9.0, which will only include containers as executables will be done later. That is all that my comment above was trying to convey.

[onionhammer]

@joperezr Never mind, I conflated this with #4739 in my head somehow

[danegsta]

A new .WithLifetime API for containers is in main and will ship with Aspire 9.0. It allows containers to be treated as persistent, in which case the AppHost won't shut them down when it stops and will attempt to re-use existing containers at launch. We track a hash of the config used to launch the container and if we detect that the current AppHost config has changed, we'll still stop and restart the container to apply the latest state (this behavior only applies to containers created by the Aspire AppHost).

A persistent container is identified by the container name; by default persistent containers are named with a combination of the service name and a postfix generated from a hash of the AppHost project path, which will make them AppHost project specific. We've also added a new .WithContainerName API to override the default container name behavior. This can be used to enable advanced scenarios with persistent containers, such as re-using a container across multiple AppHost projects.