[core] Performance improvement for runtime env serialization

[dentiny]

Addresses issue #48591

The problem is:

• If we specify anything in runtime_env in remote decorator, the parsing and serialization happens for each function invocation
  • Parsing calls parse_runtime_env, which involves a dictionary to class transformation
  • Serialization calls get_runtime_env_info, which serialize a class into json format

Discussed with @rynewang , the proposed solution here is cache pre-calculated serialized runtime env info, so the parsing and serialization only happens once at initialization.

Benchmarked with the test @jjyao mentioned on the ticket, I confirm we could reach similar performance between no env var vs with env var.

Alternatives considered:

• Use functools.cache for get_runtime_env_info, which is a stateless function
  • The caveat is, we have to figure out an acceptable way to decide whether serialization options and runtime env info is the same, simply traversing all fields is not a good plan

[rynewang]

to be more clear,

To support dynamic runtime envs in `func.option(runtime_env={...}).remote()`, we recalculate the serialized runtime env info in the `option` call. If there are multiple calls to a same option, one can save the calculation by `opt_f = func.option(runtime_env={...}); [opt_f.remote() for i in range(many)]`.

[dentiny]

I'm not sure I follow the "If there are multiple calls to a same option" part, since we don't do any caching for option calls.

[dentiny]

Adopted other comments.

[rynewang]

I think we should no longer populate updated_options["runtime_env"] ?

[dentiny]

We basically have two options:

• Keep updated_options["runtime_env"] updated, as we did in the past
• Remove runtime_env from updated_options

I choose the first method, becase updated_options corresponds to task_options and default_options, so people have expectation there's runtime_env in these options.

[rynewang]

hmm, ok since we need to support bind as well, where this optimization is not used