apache flink - Evolving TTL On The Roadmap?

Question

Flink version 1.8 introduced the support for evolving state schema. My question is will Flink introduce support of evolving time to live (TTL) to state. Right now you can't add TTL to an existing state not configured with TTL and expect to be able to restore state. I'm curious if Flink intends to open this possibility up in the future? The only work around I see might now is to enable TTL on all my state and then just set the expiration so something unbelievable high which gives me the flexibility to use the TTL features later down the line if I am still interested in it. Of course with a expiration that is based on meaningful business logic and not just some extremely high number.

Time To Live Source: https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl

Evolving State Schema Source: https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/schema_evolution.html

Answer 1

I can't recall any discussion of this on the mailing lists, nor can I find any JIRA tickets on this topic.

The workaround you've proposed isn't ideal, because you will pay the cost for state TTL (both the space for the timers and the processing effort of checking for expired state), with little benefit.

What you might do instead is to use the State Processor API to migrate your data to new state descriptors. You would use the existing state descriptor to read the data from a state snapshot (savepoint or checkpoint), and then use an updated state descriptor to write the state out to a new savepoint.