NIM Tool Calling Defect Report

This is a vendor-channel report, not a filed issue, and that is forced rather than chosen. NVIDIA publishes no public issue tracker for the NIM container image itself — nim-deploy and nim-anywhere are deployment tooling and a reference app, so neither is the right place for a defect in the image. A defect you cannot file is one you have to carry to the vendor yourself, which makes the write-up the whole deliverable: it has to stand alone for a reader who cannot ask you a follow-up question.

No internal hostnames, addresses or site paths appear below. Every container path shown is NVIDIA’s own, inside the image.

Method and evidence behind it: Diagnosing Tool Calling on an NVIDIA NIM.

Title: Tool-calling env vars (NIM_ENABLE_AUTO_TOOL_CHOICE, NIM_TOOL_CALL_PARSER, NIM_TOOL_PARSER_PLUGIN) are silently discarded — fields absent from NimAsyncEngineArgs

Summary

Tool calling cannot be enabled on this NIM by any configuration. The three documented environment variables are read, matched to known names, and then silently discarded because the corresponding fields do not exist on the engine-arguments dataclass. The only symptom is a WARNING in a wall of startup logging.

The API consequently returns HTTP 400 to every tool request:

"auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set

…even when all three variables are set on the container.

Environment

Steps to reproduce

1. Run the NIM with all three tool variables set:

-e NIM_ENABLE_AUTO_TOOL_CHOICE=1
-e NIM_TOOL_CALL_PARSER=nemotron_json
-e NIM_TOOL_PARSER_PLUGIN=<plugin>

# <plugin> is one path, wrapped here for width:
#   /opt/nim/.cache/ngc/hub
#     /models--nim--nvidia--nvidia-nemotron-nano-9b-v2
#     /snapshots/v1.2.2-ga/nemotron_toolcall_parser_no_streaming.py

2. Confirm they reached the container:

docker inspect <container> \
  --format '{{range .Config.Env}}{{println .}}{{end}}' | grep -i tool
# all three present

3. Send any request containing a tools array and "tool_choice": "auto".

Expected

The request is served and message.tool_calls is populated, or a clear startup error if the parser name is invalid.

Actual

HTTP 400: "auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set.

docker logs <container> | grep -ic tool returns 0 — the engine was never told about any of it.

Root cause

nimlib/llm/arg_utils.py applies environment overrides by attribute name:

for field_name, env_value in env_defaults.items():
    if hasattr(engine_args, field_name):
        setattr(engine_args, field_name, env_value)
    else:
        logger.warning(f"Field '{field_name}' not found in NimAsyncEngineArgs")

NimAsyncEngineArgs has 229 fields, and the tool fields are not among them:

docker exec <container> /opt/nim/.venv/bin/python - <<'EOF'
import nimlib.llm.arg_utils as a, dataclasses
f = [x.name for x in dataclasses.fields(a.NimAsyncEngineArgs)]
for k in ['enable_auto_tool_choice', 'tool_call_parser',
          'tool_parser_plugin', 'chat_template',
          'reasoning_parser', 'max_model_len']:
    print(f"{k:26} {k in f}")
EOF

{'enable_auto_tool_choice': False,
 'tool_call_parser':        False,
 'tool_parser_plugin':      False,
 'chat_template':           False,
 'reasoning_parser':        True,
 'max_model_len':           True}

The warnings are emitted at startup and by list-model-profiles:

WARNING arg_utils.py:1707] Field 'enable_auto_tool_choice' not found in NimAsyncEngineArgs
WARNING arg_utils.py:1707] Field 'tool_call_parser'       not found in NimAsyncEngineArgs
WARNING arg_utils.py:1707] Field 'tool_parser_plugin'     not found in NimAsyncEngineArgs

This appears to be an incomplete wiring rather than an intentional omission: the variable names are defined in nimlib (NIM_TOOL_CALL_PARSER_ENV_NAME etc.), and nimlib/nim_inference_sdk/vllm.py lists enable_auto_tool_choice, tool_call_parser and tool_parser_plugin in its argument table — but the dataclass those values are written to does not define them. reasoning_parser and max_model_len follow the same path and do work.

Secondary issue — no matching parser ships for this model

Even with the wiring fixed, the required parser appears to be missing:

• nemotron-3-nano’s own chat_template.jinja emits XML: <tool_call><function=NAME><parameter=KEY>value</parameter></function></tool_call>
• The only parser registered anywhere in the image is nemotron_json, and it ships as a plugin file inside a different model’s snapshot (nvidia-nemotron-nano-9b-v2)
• qwen3_coder — which matches the XML convention — is not present in the image (grep -rl qwen3_coder /opt/nim returns nothing)

Impact

Tool calling is a prerequisite for agent use. With it unavailable, this NIM cannot back any agent framework, which forces a different serving stack for agent workloads.

Suggested fix

1. Add enable_auto_tool_choice, tool_call_parser and tool_parser_plugin (and likely chat_template) to NimAsyncEngineArgs.
2. Escalate the “field not found” log line from WARNING to ERROR, or fail startup — a silently dropped configuration value is very hard to diagnose from the outside.
3. Ship a tool parser matching the model’s emitted format, or document which parser to pair with each model.