How to deploy chat prompt flow?

Question

How to deploy chat prompt flow?

Ajay Rao 0

Hi,

I have created a chat prompt flow using the Multi-Round Q&A on Your Data sample prompt flow. There are 3 vector index in the flow.

The flow works great in the test mode. However, when I deploy it as an endpoint, I am unable to consume the deployment. Can someone help me with this?

Thanks,

Ajay

1 answer

Your answer

Answer 1

Sedat SALMAN 14,180 MVP

I can recommend some troubleshooting steps

Double-check the authentication keys or tokens you're using in requests to the endpoint.
Ensure that the environment variables required for your flow, such as PRT_CONFIG_OVERRIDE for pulling connections from the workspace, are correctly set and match your Azure resources
Cross-Origin Resource Sharing (CORS) might be an issue if you're calling the endpoint from a web application. Configure CORS appropriately on your endpoint.

Also if you have some error codes please include them so we can offer more specific recommendation

Ajay Rao 0 Reputation points

2024-03-02T18:23:04.7366667+00:00

There is no error as such. I just get 'timeout of 120000ms exceeded'.

I have attached the log file for reference. The test seems to work perfectly fine.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-03-04T14:49:51.4633333+00:00

I think the deployment should be visible in your ML studio in the Azure ML workspace. Do you see any additional logs in the endpoints tab of azure ML studio.

https://ml.azure.com/endpoints

Ajay Rao 0

The log shows the following error while executing the lookup block (vector embedding):

2024-03-04 23:49:29 +0000      18 execution          WARNING  [lookup_trivia in line 0 (index starts from 0)] stderr> INFO:root:`search_function_construction` completed in 0.00017280300380662084 seconds.
2024-03-04 23:49:29 +0000      18 execution          ERROR    Node lookup_trivia in line 0 failed. Exception: Execution failure in 'lookup_trivia': (ParserError) while parsing a block mapping
  in "<unicode string>", line 2, column 3:
      api_base: https://beroundaoi.ope ... 
      ^ (line: 2)
expected <block end>, but found '<scalar>'
  in "<unicode string>", line 14, column 10:
      model: text-embedding-ada-002
             ^ (line: 14).
Traceback (most recent call last):
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/_core/flow_execution_context.py", line 190, in _invoke_tool_with_timer
    return f(**kwargs)
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/_core/tracer.py", line 458, in wrapped
    output = func(*args, **kwargs)
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow_vectordb/tool/common_index_lookup.py", line 58, in search
    search_func = _get_search_func(mlindex_content, top_k, query_type)
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow_vectordb/tool/common_index_lookup.py", line 35, in _get_search_func
    mlindex_config = yaml.load(mlindex_content)
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/main.py", line 451, in load
    return constructor.get_single_data()
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 114, in get_single_data
    node = self.composer.get_single_node()
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/composer.py", line 71, in get_single_node
    if not self.parser.check_event(StreamEndEvent):
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/parser.py", line 141, in check_event
    self.current_event = self.state()
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/parser.py", line 614, in parse_block_mapping_key
    raise ParserError(
ruamel.yaml.parser.ParserError: while parsing a block mapping
  in "<unicode string>", line 2, column 3:
      api_base: https://beroundaoi.ope ... 
      ^ (line: 2)
expected <block end>, but found '<scalar>'
  in "<unicode string>", line 14, column 10:
      model: text-embedding-ada-002
             ^ (line: 14)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/_core/flow_execution_context.py", line 88, in invoke_tool
    result = self._invoke_tool_with_timer(node, f, kwargs)
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/_core/flow_execution_context.py", line 201, in _invoke_tool_with_timer
    raise ToolExecutionError(node_name=node_name, module=module) from e
promptflow._core._errors.ToolExecutionError: Execution failure in 'lookup_trivia': (ParserError) while parsing a block mapping
  in "<unicode string>", line 2, column 3:
      api_base: https://beroundaoi.ope ... 
      ^ (line: 2)
expected <block end>, but found '<scalar>'
  in "<unicode string>", line 14, column 10:
      model: text-embedding-ada-002
             ^ (line: 14)
2024-03-04 23:49:29 +0000      18 execution          WARNING  [lookup_reviews in line 0 (index starts from 0)] stderr> INFO:root:`search_function_construction` completed in 0.004915461002383381 seconds.
2024-03-04 23:49:29 +0000      18 execution          ERROR    Node lookup_reviews in line 0 failed. Exception: Execution failure in 'lookup_reviews': (IndexError) string index out of range.
Traceback (most recent call last):
  File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/ruamel/yaml/reader.py", line 124, in peek
    return self.buffer[self.pointer + index]
IndexError: string index out of range

Share via

How to deploy chat prompt flow?

1 answer

Your answer