![\"Debugging](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Phoenix-Tracing-And-Debugging-LLMs-like-a-Pro.png\"\/)
![\"Debugging](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/Phoenix-Tracing-And-Debugging-LLMs-like-a-Pro.png\")
Picture by Creator | Canva<\/span><\/center>
\n\u00a0<\/p>\n
#\u00a0<\/span>Introduction<\/h2>\n\u00a0
Conventional debugging with print()<\/code> or logging works, however it\u2019s sluggish and clunky with LLMs. Phoenix gives a timeline view of each step, immediate, and response inspection, error detection with retries, visibility into latency and prices, and an entire visible understanding of your app. Phoenix by Arize AI is a robust open-source observability and tracing software particularly designed for LLM functions. It helps you monitor, debug, and hint every little thing occurring in your LLM pipelines visually. On this article, we\u2019ll stroll by what Phoenix does and why it issues, tips on how to combine Phoenix with LangChain step-by-step, and tips on how to visualize traces within the Phoenix UI.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>What’s Phoenix?<\/h2>\n\u00a0
Phoenix is an open-source observability and debugging software made for big language mannequin functions. It captures detailed telemetry knowledge out of your LLM workflows, together with prompts, responses, latency, errors, and gear utilization, and presents this info in an intuitive, interactive dashboard. Phoenix permits builders to deeply perceive how their LLM pipelines behave contained in the system, establish and debug points with immediate outputs, analyze efficiency bottlenecks, monitor utilizing tokens and related prices, and hint any errors\/retry logic throughout execution part. It helps constant integrations with fashionable frameworks like LangChain and LlamaIndex, and likewise presents OpenTelemetry assist for extra personalized setups.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>Step-by-Step Setup<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>1. Putting in Required Libraries<\/h4>\nEnsure you have Python 3.8+ and set up the dependencies:<\/p>\n
\npip set up arize-phoenix langchain langchain-together openinference-instrumentation-langchain langchain-community<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>2. Launching Phoenix<\/h4>\nAdd this line to launch the Phoenix dashboard:<\/p>\n
\nimport phoenix as px
\npx.launch_app()<\/code><\/pre>\n<\/div>\n\u00a0
This begins an area dashboard at http:\/\/localhost:6006<\/a>.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>3. Constructing the LangChain Pipeline with Phoenix Callback<\/h4>\nLet\u2019s perceive Phoenix utilizing a use case. We’re constructing a easy LangChain-powered chatbot. Now, we wish to:<\/p>\n
\n- Debug if the immediate is working<\/li>\n
- Monitor how lengthy the mannequin takes to reply<\/li>\n
- Monitor immediate construction, mannequin utilization, and outputs<\/li>\n
- See all this visually as an alternative of logging every little thing manually<\/li>\n<\/ul>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 1: Launch the Phoenix Dashboard within the Background<\/h4>\n\nimport threading
\nimport phoenix as px
\n
\n# Launch Phoenix app regionally (entry at http:\/\/localhost:6006)
\ndef run_phoenix():
\n px.launch_app()
\n
\nthreading.Thread(goal=run_phoenix, daemon=True).begin()<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 2: Register Phoenix with OpenTelemetry & Instrument LangChain<\/h4>\n\nfrom phoenix.otel import register
\nfrom openinference.instrumentation.langchain import LangChainInstrumentor
\n
\n# Register OpenTelemetry tracer
\ntracer_provider = register()
\n
\n# Instrument LangChain with Phoenix
\nLangChainInstrumentor().instrument(tracer_provider=tracer_provider)<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 3: Initialize the LLM (Collectively API)<\/h4>\n\nfrom langchain_together import Collectively
\n
\nllm = Collectively(
\n mannequin=\"meta-llama\/Llama-3-8b-chat-hf\",
\n temperature=0.7,
\n max_tokens=256,
\n together_api_key=\"your-api-key\", # Substitute together with your precise API key
\n)<\/code><\/pre>\n<\/div>\n\u00a0
Please don\u2019t neglect to interchange the \u201cyour-api-key\u201d together with your precise collectively.ai API key. You will get it utilizing this hyperlink<\/a>.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 4: Outline the Immediate Template<\/h4>\n\nfrom langchain.prompts import ChatPromptTemplate
\n
\nimmediate = ChatPromptTemplate.from_messages([
\n (\"system\", \"You are a helpful assistant.\"),
\n (\"human\", \"{question}\"),
\n])<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 5: Mix Immediate and Mannequin right into a Chain<\/h4>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 6: Ask A number of Questions and Print Responses<\/h4>\n\nquestions = [
\n \"What is the capital of France?\",
\n \"Who discovered gravity?\",
\n \"Give me a motivational quote about perseverance.\",
\n \"Explain photosynthesis in one sentence.\",
\n \"What is the speed of light?\",
\n]
\n
\nprint(\"Phoenix operating at http:\/\/localhost:6006n\")
\n
\nfor q in questions:
\n print(f\" Query: {q}\")
\n response = chain.invoke({\"query\": q})
\n print(\" Reply:\", response, \"n\")<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 7: Hold the App Alive for Monitoring<\/h4>\n\nattempt:
\n whereas True:
\n move
\nbesides KeyboardInterrupt:
\n print(\" Exiting.\")<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
#\u00a0<\/span>Understanding Phoenix Traces & Metrics<\/h2>\n\u00a0
Earlier than seeing the output, we must always first perceive Phoenix metrics. You have to to first perceive what traces and spans are:
Hint:<\/strong> Every hint represents one full run of your LLM pipeline. For instance, every query like \u201cWhat’s the capital of France?\u201d generates a brand new hint.
Spans:<\/strong> Every hint is combined of a number of spans, every representing a stage in your chain:<\/p>\n\n- ChatPromptTemplate.format: Immediate formatting<\/li>\n
- TogetherLLM.invoke: LLM name<\/li>\n
- Any customized elements you add<\/li>\n<\/ul>\n
Metrics Proven per Hint<\/strong>
\u00a0<\/p>\n
Conventional debugging with
print()<\/code> or logging works, however it\u2019s sluggish and clunky with LLMs. Phoenix gives a timeline view of each step, immediate, and response inspection, error detection with retries, visibility into latency and prices, and an entire visible understanding of your app. Phoenix by Arize AI is a robust open-source observability and tracing software particularly designed for LLM functions. It helps you monitor, debug, and hint every little thing occurring in your LLM pipelines visually. On this article, we\u2019ll stroll by what Phoenix does and why it issues, tips on how to combine Phoenix with LangChain step-by-step, and tips on how to visualize traces within the Phoenix UI.<\/p>\n\u00a0<\/p>\n
#\u00a0<\/span>What’s Phoenix?<\/h2>\n\u00a0
Phoenix is an open-source observability and debugging software made for big language mannequin functions. It captures detailed telemetry knowledge out of your LLM workflows, together with prompts, responses, latency, errors, and gear utilization, and presents this info in an intuitive, interactive dashboard. Phoenix permits builders to deeply perceive how their LLM pipelines behave contained in the system, establish and debug points with immediate outputs, analyze efficiency bottlenecks, monitor utilizing tokens and related prices, and hint any errors\/retry logic throughout execution part. It helps constant integrations with fashionable frameworks like LangChain and LlamaIndex, and likewise presents OpenTelemetry assist for extra personalized setups.<\/p>\n
\u00a0<\/p>\n
#\u00a0<\/span>Step-by-Step Setup<\/h2>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>1. Putting in Required Libraries<\/h4>\nEnsure you have Python 3.8+ and set up the dependencies:<\/p>\n
\npip set up arize-phoenix langchain langchain-together openinference-instrumentation-langchain langchain-community<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>2. Launching Phoenix<\/h4>\nAdd this line to launch the Phoenix dashboard:<\/p>\n
\nimport phoenix as px
\npx.launch_app()<\/code><\/pre>\n<\/div>\n\u00a0
This begins an area dashboard at http:\/\/localhost:6006<\/a>.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>3. Constructing the LangChain Pipeline with Phoenix Callback<\/h4>\nLet\u2019s perceive Phoenix utilizing a use case. We’re constructing a easy LangChain-powered chatbot. Now, we wish to:<\/p>\n
\n- Debug if the immediate is working<\/li>\n
- Monitor how lengthy the mannequin takes to reply<\/li>\n
- Monitor immediate construction, mannequin utilization, and outputs<\/li>\n
- See all this visually as an alternative of logging every little thing manually<\/li>\n<\/ul>\n
\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 1: Launch the Phoenix Dashboard within the Background<\/h4>\n\nimport threading
\nimport phoenix as px
\n
\n# Launch Phoenix app regionally (entry at http:\/\/localhost:6006)
\ndef run_phoenix():
\n px.launch_app()
\n
\nthreading.Thread(goal=run_phoenix, daemon=True).begin()<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 2: Register Phoenix with OpenTelemetry & Instrument LangChain<\/h4>\n\nfrom phoenix.otel import register
\nfrom openinference.instrumentation.langchain import LangChainInstrumentor
\n
\n# Register OpenTelemetry tracer
\ntracer_provider = register()
\n
\n# Instrument LangChain with Phoenix
\nLangChainInstrumentor().instrument(tracer_provider=tracer_provider)<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 3: Initialize the LLM (Collectively API)<\/h4>\n\nfrom langchain_together import Collectively
\n
\nllm = Collectively(
\n mannequin=\"meta-llama\/Llama-3-8b-chat-hf\",
\n temperature=0.7,
\n max_tokens=256,
\n together_api_key=\"your-api-key\", # Substitute together with your precise API key
\n)<\/code><\/pre>\n<\/div>\n\u00a0
Please don\u2019t neglect to interchange the \u201cyour-api-key\u201d together with your precise collectively.ai API key. You will get it utilizing this hyperlink<\/a>.<\/p>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 4: Outline the Immediate Template<\/h4>\n\nfrom langchain.prompts import ChatPromptTemplate
\n
\nimmediate = ChatPromptTemplate.from_messages([
\n (\"system\", \"You are a helpful assistant.\"),
\n (\"human\", \"{question}\"),
\n])<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 5: Mix Immediate and Mannequin right into a Chain<\/h4>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 6: Ask A number of Questions and Print Responses<\/h4>\n\nquestions = [
\n \"What is the capital of France?\",
\n \"Who discovered gravity?\",
\n \"Give me a motivational quote about perseverance.\",
\n \"Explain photosynthesis in one sentence.\",
\n \"What is the speed of light?\",
\n]
\n
\nprint(\"Phoenix operating at http:\/\/localhost:6006n\")
\n
\nfor q in questions:
\n print(f\" Query: {q}\")
\n response = chain.invoke({\"query\": q})
\n print(\" Reply:\", response, \"n\")<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
\/\/\u00a0<\/span>Step 7: Hold the App Alive for Monitoring<\/h4>\n\nattempt:
\n whereas True:
\n move
\nbesides KeyboardInterrupt:
\n print(\" Exiting.\")<\/code><\/pre>\n<\/div>\n\u00a0<\/p>\n
#\u00a0<\/span>Understanding Phoenix Traces & Metrics<\/h2>\n\u00a0
Earlier than seeing the output, we must always first perceive Phoenix metrics. You have to to first perceive what traces and spans are:
Hint:<\/strong> Every hint represents one full run of your LLM pipeline. For instance, every query like \u201cWhat’s the capital of France?\u201d generates a brand new hint.
Spans:<\/strong> Every hint is combined of a number of spans, every representing a stage in your chain:<\/p>\n\n- ChatPromptTemplate.format: Immediate formatting<\/li>\n
- TogetherLLM.invoke: LLM name<\/li>\n
- Any customized elements you add<\/li>\n<\/ul>\n
Metrics Proven per Hint<\/strong>
\u00a0<\/p>\n
![\"Phoenix](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/image-1-1.png\"\/)
![\"Phoenix](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/image-1-1.png\")
![\"Phoenix](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/image-2-1.png\"\/)
![\"Phoenix](\"https:\/\/www.kdnuggets.com\/wp-content\/uploads\/image-2-1.png\")
<\/p>\n