All-in-one observability platform New Relic is integrating its platform with NVIDIA NIM inference microservices to reduce the complexity and costs of developing, deploying, and monitoring generative AI apps.
Customers can now use New Relic AI monitoring to gain visibility across the AI stack for applications built with NVIDIA NIM, all with a simplified setup and while ensuring data security. This complements the robust security features and ease of use of NVIDIA NIM’s self-hosted models, which accelerates generative AI application delivery.
“We are pioneering AI observability by extending our platform to include AI apps built with NVIDIA NIM,” said New Relic CEO Ashan Willy. “Combining NVIDIA’s AI technology with our expertise in observability and APM gives enterprises a competitive edge in the AI race.”
Key features and use cases for AI monitoring include:
- Full AI stack visibility: The ability to spot issues faster with a holistic view across apps, NVIDIA GPU-based infrastructure, AI layer, response quality, token count, and APM golden signals.
- Deep trace insights for every response: Resolve performance and quality issues like bias, toxicity, and hallucinations by tracing the entire lifecycle of AI responses
- Model inventory: Easily isolate model-related performance, error, and cost issues by tracking key metrics across all NVIDIA NIM inference microservices in one place
- Model comparison: Compare the performance of NVIDIA NIM inference microservices running in production in a single view to optimise model choice based on infrastructure and user needs.
- Deep GPU insights: Analyse critical accelerated computing metrics such as GPU utilisation, temperature, and performance states; understand context and resolve problems faster.
-
Enhanced data security: In addition to NVIDIA’s self-hosted model’s security advantage, New Relic allows customers to exclude monitoring of sensitive data (PII) in their AI requests and responses.
This integration follows New Relic’s recent addition to NVIDIA’s AIOps partner ecosystem. New Relic AI monitoring is available as part of its all-in-one observability platform and offered via its usage-based pricing model.
