InterSystems Expands IRIS Data Platform with Vector Search to Support Next-Gen AI Applications

March 26, 2024 — In this blog post released today, Scott Gnau, head of Global Data Platforms at InterSystems announced the addition of vector search to the company’s IRIS data platform. Vector search will make massive, disparate datasets easily searchable, especially for GenAI tasks related to natural language processing, text or image analysis. For developers, this will make it easier for them to create more powerful GenAI applications on the IRIS platform.

Artificial intelligence has transformative potential for driving value and insights from data. As we progress toward a world where nearly every application will be AI-driven, developers building those applications will need the right tools for creating experiences from these applications. This is why InterSystems is excited to announce the addition of vector search to the IRIS data platform.

Tools like vector search are essential for enabling efficient and accurate retrieval of relevant information from massive datasets when working with large language models. By converting text and images into high-dimensional vectors, these techniques allow quick comparisons and searches, even when dealing with millions of files from disparate datasets across the organization.

InterSystems IRIS Data Platform Delivers a Unified Foundation for Next Generation Applications

At InterSystems, we are always looking for ways to bring next generation data processing as close to our customers’ data as possible without having to transfer data to specialized systems. By adding vector search to the InterSystems IRIS data platform, we are making the data platform searchable through vector embeddings in order to enhance the functionality of the software for tasks related to natural language processing (NLP), text, and image analysis.

This integration will make it easier for developers to create applications that use generative AI to complete complex tasks for a wide range of use cases and deliver up-to-date responses based on proprietary data processed by InterSystems. It also means they can do this with very curated data while being confident in keeping their internal proprietary intelligence secure.

Scott Gnau, Head of Global Data Platforms at InterSystems

This capability allows the InterSystems IRIS data platform to manage and query content and related dense vector embeddings, particularly as it enables RAG integration to develop generative AI-based applications. With the rapidly evolving toolsets available, seamless RAG integration allows agile adoption for new models and use cases.

What Benefits Does This Technology Bring to Customers?

BioStrand, an AI-powered drug discovery company, is part of the InterSystems Innovation Program that helps start-ups build applications on our IRIS platform. At the core of BioStrand’s offerings is the Lens AI platform, a versatile solution that supports various applications including antibody drug discovery and design. By leveraging advanced algorithms, Lens AI can swiftly identify and craft novel drug compounds, significantly reducing R&D timelines from development to commercialization. This model uniquely combines the strengths of Large Language Models (LLMs) using an advanced stacking technique with BioStrand’s patented HYFT Technology.

HYFTs are a type of embedding that serve as unique ‘fingerprints’ in biological sequences, enabling BioStrand to assign embeddings from different LLMs with high precision. This foundation model represents a vast and continuously expanding knowledge graph, mapping an impressive 25 billion relationships across 660 million data objects. This comprehensive graph interconnects sequences, structures, functions from the entire biosphere, together with bibliographic information. It also incorporates cutting-edge technologies like Retrieval Augmented Generation, SQL vector search and generative capabilities of LLMs together with the semantic expressiveness of knowledge graphs.

Vector Search Will Fundamentally Change How Developers Interact with IRIS

We’re just scratching the surface when it comes to implementing this technology. We’ll be sharing more customer stories as vector search changes how they interact with their data and what new AI applications are developed with vector search.

Our commitment to maintaining the highest standards of privacy, security, and responsibility will guide a thoughtful and just approach to AI that creates trust while accelerating innovation, ensuring customer success, and demonstrating a commitment to excellence. We believe that transparency, responsibility and explainability are key to establishing trust in and driving innovation from AI systems.

Source: Scott Gnau, InterSystems