The Blog

today, the integrative computational analysis of multi-omics data has become a central tenet of the big data-driven approach to biological research. and yet, there is still a lack of gold standards when it comes to evaluating and classifying integration methodologies that can be broadly applied across multi-omics analysis. more importantly, the lack of a cohesive or universal approach to big data integration is also creating new challenges in the development of novel computational approaches for multi-omics analysis. one aspect of sequence search and comparison, however, has not changed much at all – a biological sequence in a predefined and acceptable data format is still the primary input in most research. this approach is probably and arguably valid in many if not most real-world research scenarios. take machine learning (ml) models, for instance, which are increasingly playing a central role in the analysis of genomic big data. biological data presents several unique challenges, such as missing values and precision variations across omics modalities, that simply expand the gamut of integration strategies required to address each specific challenge. for example, omics datasets often contain missing values, which can hamper downstream integrative bioinformatics analyses. this requires an additional imputation process to infer the missing values in these incomplete datasets before statistical analyses can be applied. then there is the high-dimension low sample size (hdlss) problem, where the variables significantly outnumber samples, leading ml algorithms to overfit these datasets, thereby decreasing their generalisability on new data. in addition, there are multiple challenges inherent to all biological data irrespective of analytical methodology or framework. to start with there is the sheer heterogeneity of omics data comprising a variety of datasets originating from a range of data modalities and comprising completely different data distributions and types that have to be handled appropriately. integrating heterogeneous multi-omics data presents a cascade of challenges involving the unique data scaling, normalisation, and transformation requirements of each individual dataset. any effective integration strategy will also have to account for the regulatory relationships between datasets from different omics layers in order to accurately and holistically reflect the nature of this multidimensional data. furthermore, there is the issue of integrating omics and non-omics (ono) data, like clinical, epidemiological or imaging data, for example, in order to enhance analytical productivity and to access richer insights into biological events and processes. currently, the large-scale integration of non-omics data with high-throughput omics data is extremely limited due to a range of factors, including heterogeneity and the presence of subphenotypes, for instance. the crux of the matter is that without effective and efficient data integration, multi-omics analysis will only tend to become more complex and resource-intensive without any proportional or even significant augmentation in productivity, performance, or insight generation. an overview of multi-omics data integration early approaches to multi-omics analysis involved the independent analysis of different data modalities and combining results for a quasi-integrated view of molecular interactions. but the field has evolved significantly since then into a broad range of novel, predominantly algorithmic meta-analysis frameworks and methodologies for the integrated analysis of multi-dimensional multi-omics data. however, the topic of data integration and the challenges involved is often overshadowed by the ground-breaking developments in integrated, multi-omics analysis. it is therefore essential to understand the fundamental conceptual principles, rather than any specific method or framework, that define multi-omics data integration. horizontal vs vertical data integration multi-omics datasets are broadly organized as horizontal or vertical, corresponding to the complexity and heterogeneity of multi-omics data. horizontal datasets are typically generated from one or two technologies, for a specific research question and from a diverse population, and represent a high degree of real-world biological and technical heterogeneity. horizontal or homogeneous data integration, therefore, involves combining data from across different studies, cohorts or labs that measure the same omics entities. vertical data refers to data generated using multiple technologies, probing different aspects of the research question, and traversing the possible range of omics variables including the genome, metabolome, transcriptome, epigenome, proteome, microbiome, etc. vertical, or heterogeneous, data integration involves multi-cohort datasets from different omics levels, measured using different technologies and platforms. the fact that vertical integration techniques cannot be applied for horizontal integrative analysis and vice-versa opens up an opportunity for conceptual innovation in multi-omics for data integration techniques that can enable an integrative analysis of both horizontal and vertical multi-omics datasets. of course, each of these broad data heads can further be broken down into a range of approaches based on utility and efficiency. 5 integration strategies for vertical data a 2021 mini-review of general approaches to vertical data integration for ml analysis defined five distinct integration strategies – early, mixed, intermediate, late and hierarchical – based not just on the underlying mathematics but on a variety of factors including how they were applied. here’s a quick rundown of each approach. early integration is a simple and easy-to-implement approach that concatenates all omics datasets into a single large matrix. this increases the number of variables, without altering the number of observations, which results in a complex, noisy and high dimensional matrix that discounts dataset size difference and data distribution. mixed integration addresses the limitations of the early model by separately transforming each omics dataset into a new representation and then combining them for analysis. this approach reduces noise, dimensionality, and dataset heterogeneities. intermediate integration simultaneously integrates multi-omics datasets to output multiple representations, one common and some omics-specific. however, this approach often requires robust pre-processing due to potential problems arising from data heterogeneity. late integration circumvents the challenges of assembling different types of omics datasets by analysing each omics separately and combining the final predictions. this multiple single-omics approach does not capture inter-omics interactions. hierarchical integration focuses on the inclusion of prior regulatory relationships between different omics layers so that analysis can reveal the interactions across layers. though this strategy truly embodies the intent of trans-omics analysis, this is still a nascent field with many hierarchical methods focusing on specific omics types, thereby making them less generalisable. the availability of an unenviable choice of conceptual approaches – each with its own scope and limitations in terms of throughput, performance, and accuracy – to multi-omics data integration represents one of the biggest bottlenecks to downstream analysis and biological innovation. researchers often spend more time mired in the tedium of data munging and wrangling than they do extracting knowledge and novel insights. most conventional approaches to data integration, moreover, seem to involve some form of compromise involving either the integrity of high-throughput multi-omics data or achieving true trans-omics analysis. there has to be a new approach to multi-omics data integration that can 1), enable the one-click integration of all omics and non-omics data, and 2), preserve the biological consistency, in terms of correlations and associations across different regulatory datasets, for integrative multi-omics analysis in the process. the mindwalk hyft model for data integration at biostrand we took a lateral approach to the challenge of biological data integration. rather than start with a technological framework that could be customised for the complexity and heterogeneity of multi-omics data, we set out to decode the atomic units of all biological information that we call hyfts™. hyfts are essentially the building blocks of biological information, which means that they enable the tokenisation of all biological data, irrespective of species, structure, or function, to a common omics data language. we then built the framework to identify, collate, and index hyfts from sequence data. this enabled us to create a proprietary pangenomic knowledge database of over 660 million hyfts, each containing comprehensive information about variation, mutation, structure, etc., from over 450 million sequences available across 12 popular public databases. with the mindwalk platform, researchers and bioinformaticians have instant access to all the data from some of the most widely used omics data sources. plus, our unique hyfts framework allows researchers the convenience of one-click normalization and integration of all their proprietary omics data and metadata. based on our biological discovery, we were able to normalise and integrate all publicly available omics data, including patent data, at scale, and render them multi-omics analysis-ready. the same hyft ip can also be applied to normalise and integrate proprietary omics data. the transversal language of hyfts enables the instant normalisation and integration of multi-omics research-relevant data and metadata into one single source of truth. with the mindwalk approach to multi-omic data integration, it is no longer about whether research data is horizontal or vertical, homogeneous or heterogeneous, text or sequence, omics or non-omics. if it is data that is relevant to your research, mindwalk enables you to integrate it with just one click.

conventional vaccine development, still based predominantly on systems developed in the last century, is a complex process that takes between 10-15 years on average. until the covid-19 pandemic, when two mrna vaccines went from development to deployment in less than a year, the record for the fastest development of a new vaccine, in just four years, had gone unchallenged for over half a century. this revolutionary boost to the vaccine development cycle stemmed from two uniquely 21st century developments: first, the access to cost-effective next-generation sequencing technologies with significantly enhanced speed, coverage and accuracy that enabled the rapid sequencing of the sars-cov-2 virus. and second, the availability of innovative state-of-the-art bioinformatics technologies to convert raw data into actionable insights, without which ngs would have just resulted in huge stockpiles of dormant or dark data. in the case of covid-19, cutting edge bioinformatics approaches played a critical role in enabling researchers to quickly hone in on the spike protein gene as the vaccine candidate. ngs technologies and advanced bioinformatics solutions have been pivotal to mitigate the global impact of covid-19, providing the tools required for detection, tracking, containment and treatment, the identification of biomarkers, the discovery of potential drug targets, drug repurposing, and exploring other therapeutic opportunities. however, the combination of gene engineering and information technologies is already creating the foundation for the fourth generation of sequencing technologies for faster and more cost-effective whole-genome sequencing and disease diagnosis. as a result, continuous innovation has become an evolutionary imperative for modern bioinformatics as it has to keep up with the developmental pace of ngs technologies and accelerate the transformation of an exponentially increasing trove of data into knowledge. however, the raw volume and velocity of data sequences is just one facet of big data genomics. today, bioinformatics solutions have to cope with a variety of complex data, in heterogeneous formats, from diverse data sources, from different sequencing methods connected to different -omes, and relating to different characteristics of genomes. more importantly, the critical focus of next-generation bioinformatics technologies has to be on catalysing new pathways and dimensions in biological research that can drive transformative change in precision medicine and public health. in the following section, we look at the current evolutionary trajectory of bioinformatics in the context of three key omics analysis milestones. three key milestones in the evolution of bioinformatics the steady evolution of bioinformatics over the past two decades into a cross-disciplinary and advanced computational practice has enabled several noteworthy milestones in omics analysis. the following, however, are significant as they best showcase the growth and expansion of omics research across multiple biological layers and dimensions, all made possible by a new breed of bioinformatics solutions. searching and aligning sequences are in its essence a problem of matching letters on a grid and assigning regions of high similarity versus regions of high variation. but nature has done a great deal to make this a challenging task. integrated multi-omics for years, omics data has provided the requisite basis for the molecular characterisation of various diseases. however, genomic studies of diseases, like cancer for example, invariably include data from heterogeneous data sources and understanding cross-data associations and interactions can reveal deep molecular insights into complex biological processes that may simply not be possible with single-source analysis. combining data across metabolomics, genomics, transcriptomics, and proteomics can reveal hidden associations and interactions between omics variables, elucidate the complex relationships between molecular layers and enable a holistic, pathway-oriented view of biology. an integrated and unified approach to multiple omics analysis has a range of novel applications in the prediction, detection, and prevention of various diseases, in drug discovery, and in designing personalised treatments. and, thanks to the development of next-generation bioinformatics platforms, it is now possible to integrate not just omics data but all types of relevant medical, clinical, and biological data, both structured and unstructured, under a unified analytical framework for a truly integrated approach to multi-omics analysis. single-cell multi-omics where multi-omics approaches focus on the interactions between omics layers to clarify complex biological processes, single-cell multi-omics enable the simultaneous and comprehensive analysis of the unique genotypic and phenotypic characteristics of single cells as well as the regulatory mechanisms that are evident only at single-cell resolution. earlier approaches to single-cell analysis involved the synthesis of data from individual cells and then computationally linking different modalities across cells. but with next-generation multi-omics technologies, it is now possible to directly look at each cell in multiple ways and perform multiple analyses at the single-cell level. today, advanced single-cell multi-omics technologies can measure a wide range of modalities, including genomics, transcriptomics, epigenomics, and proteomics, to provide ground-breaking insights into cellular phenotypes and biological processes. best-in-class solutions provide the framework required to seamlessly integrate huge volumes of granular data across multiple experiments, measurements, cell types, and organisms, and facilitate the integrative and comprehensive analysis of single-cell data. spatial transcriptomics single-cell rna sequencing enabled a more fine-grained assessment of each cell’s transcriptome. however, single-cell sequencing techniques are limited to tissue-dissociated cells that have lost all spatial information. delineating the positional context of cell types within a tissue is important for several reasons, including the need to understand the chain of information between cells in a tissue, to correlate cell groups and cellular functions, and to identify cell distribution differences between normal and diseased cells. spatial single-cell transcriptomics, or spatialomics, considered to be the next wave after single-cell analysis, combines imaging and single-cell sequencing to map the position of particular transcripts on a tissue, thereby revealing where particular genes are expressed and indicating the functional context of individual cells. even though many bioinformatics capabilities for the analysis of single-cell rna-seq data are shared with spatially resolved data, analysis pipelines diverge at the level of the quantification matrix, requiring specialised tools to extract knowledge from spatial data. however, there are advanced analytics platforms that use a unique single data framework to ingest all types of data, including spatial coordinates, for integrated analysis. quo vadis, bioinformatics? bioinformatics will continue to evolve alongside, if not ahead of, emerging needs and opportunities in biological research. but if there is one key takeaway from the examples cited here, it is that a reductionist approach – one that is limited to a single omics modality or discipline or even dimension – yields limited and often suboptimal results. if bioinformatics is to continue driving cutting edge biological research to tackle some of the most complex questions of our times, then the focus needs to be on developing a more holistic, systems bioinformatics approach to analysis. bioinformatics systems biology analysis is not an entirely novel concept, though its application is not particularly commonplace. but systems bioinformatics applies a well-defined systems approach framework to the entire spectrum of omics data with the emphasis on defining the level of resolution and the boundary of the system of interest in order to study the system as a whole, rather than as a sum of its components. the focus is on combining the bottom-up approach of systems biology with the data-driven top-down approach of classical bioinformatics to integrate different levels of information. the advent of multi-omics has, quite paradoxically, only served to accentuate the inherently siloed nature of omics approaches. even though the pace of bioinformatics innovations has picked up over the past couple of decades, the broader practice itself is still mired in a fragmented multiplicity of domain, project, or data specific solutions and pipelines. there is still a dearth of integrated end-to-end solutions with the capabilities to integrate multi-modal datasets, scale effortlessly from the study of specific molecular mechanisms to system-wide analysis of biological systems, and empower collaboration across disciplines research communities. integration at scale and across disciplines, datasets, sources, and computational methodologies is now the grand challenge for bioinformatics and represents the first step towards a future of systems bioinformatics.

the exponential generation of data by modern high-throughput, low-cost next generation sequencing (ngs) technologies is set to revolutionise genomics and molecular biology and enable a deeper and richer understanding of biological systems. and it is not just about more volumes of highly accurate, multi-layered data. it’s also about more types of omics datasets, such as glycomics, lipidomics, microbiomics, and phenomics. the increasing availability of large-scale, multidimensional and heterogeneous datasets has the potential to open up new insights into biological systems and processes, improve and increase diagnostic yield, and pave the way to shift from reductionist biology to a more holistic systems biology approach to decoding the complexities of biological entities. it has already been established that multi-dimensional analysis – as opposed to single layer analyses – yields better results from a statistical and a biological point of view, and can have a transformative impact on a range of research areas, such as genotype-phenotype interactions, disease biology, systems microbiology, and microbiome analysis. however, applying systems thinking principles to biological data requires the development of radically new integrative techniques and processes that can enable the multi-scale characterisation of biological systems. combining and integrating diverse types of omics data from different layers of biological regulation is the first computational challenge – and the next big opportunity – on the way to enabling a unified end-to-end workflow that is truly multi-omics. the challenge is quite colossal – indeed, a 2019 article in the journal of molecular endocrinology refers to the successful implementation of more than two datasets as very rare. data integration challenges in multi-omics analysing omics datasets at just one level of biological complexity is challenging enough. multi-omics analysis amplifies those challenges and introduces some unfamiliar new complications around data integration/fusion, clustering, visualisation, and functional characterisation. for instance, accommodating for the inherent complexity of biological systems, the sheer number of biological variables and the relatively low number of biological samples can on its own turn out to be a particularly difficult assignment. over and above this, there is a litany of other issues including process variations in data cleaning and normalisation, data dimensionality reduction, biological contextualisation, biomolecule identification, statistical validation, etc., etc., etc. data heterogeneity, arguably the raison d'être for integrated omics, is often the primary hurdle in multi-omics data management. omics data is typically distributed across multiple silos defined by domain, type, and access type (public/proprietary), to name just a few variables. more often than not, there are significant variations between datasets in terms of the technologies/platforms that were used to generate these datasets, nomenclature, data modalities, assay types, etc. data harmonisation, therefore, becomes a standard pre-integration process. but the process for data scaling, data normalisation, and data transformation to harmonise data can vary across different dataset types and sources. for example, there is a difference between normalisation and scaling techniques between rna-seq datasets and small rna-seq datasets. multi-omics data integration has its own set of challenges, including lack of reliability in parameter estimation, preserving accuracy in statistical inference, and/or the prevalence of large standard errors. there are, however, several tools currently available for multi-omics data integration, though they come with their own limitations. for example, there are web-based tools that require no computational experience – but the lack of visibility into their underlying processes makes it a challenge to deploy them for large-scale scientific research. on the other end of the spectrum, there are more sophisticated tools that afford more customisation and control – but also require considerable expertise in computational techniques. in this context, the development of a universal standard or unified framework for pre-analysis, let alone an integrated end-to-end pipeline for multi-omics analysis, seems rather daunting. however, if multi-omics analysis is to yield diagnostic value at scale, it is imperative that it quickly evolves from being a dispersed syndicate of tools, techniques and processes to a new integrated multi-omics paradigm that is versatile, computationally feasible and user-friendly. a platform approach to multi-omics analysis the data integration challenge in multi-omics essentially boils down to this. there either has to be a technological innovation designed specifically to handle the fine-grained and multidimensional heterogeneity of biological data. or, there has to be a biological discovery that unifies all omics data and makes them instantly computable even for conventional technologies. at mindwalk, we took the latter route and came up with hyfts™, a biological discovery that can instantly make all omics data computable. normalising/integrating data with hyfts™ we started with a new technique for indexing cellular blueprints and building blocks and used it to identify and catalogue unique signature sequences, or biological fingerprints, in dna, rna, and aa that we call hyft™ patterns. each hyft™ comprises multiple layers of information, relating to function, structure, position, etc., that together create a multilevel information network. we then designed a mindwalk parser to identify, collate and index hyfts™ from over 450 million sequences available across 11 popular public databases. this helped us create a proprietary pangenomic knowledge database using over 660 million hyft™ patterns containing information about variation, mutation, structure, and more. based on our biological discovery, we were able to normalise and integrate all publicly available omics data, including patent data, at scale, and render them multi-omics analysis-ready. the same hyft™ ip can also be applied to normalise and integrate proprietary omics data. making 660 million data points accessible that’s a lot of data points. so, we made it searchable. with google-like advanced indexing and exact matching technologies, only exact matches to search inputs are returned. through a simple search interface – use plain text or a fasta file – researchers can now accurately retrieve all relevant information about sequence alignments, similarities, and differences from a centralised knowledge base with information on millions of organisms in just 3 seconds. synthesising knowledge with our ai-powered saas platform around these core capabilities, we built the mindwalk saas platform with state-of-the-art ai tools to expand data management capabilities, mitigate data complexity, and to empower researchers to intuitively synthesise knowledge out of petabytes of biological data. with our platform, researchers can easily add different types of structured and unstructured data, leverage its advanced graph-based data mining features to extract insights from huge volumes of data, and use built-in genomic analysis tools for annotation and variation analysis. multi-omics as a platform as omics data sets become more multi-layered and multidimensional, only a truly sequence integrated multi-omics analysis solution can enable the discovery of novel and practically beneficial biological insights. with mindwalk platform, delivered as a saas, we believe we have created an integrated platform that enables a user-friendly, automated, intelligent, and data-ingestion-to-insight approach to multi-omics analysis. it eliminates all the data management challenges associated with conventional multi-omics analysis solutions and offers a cloud-based platform-centric approach to multi-omics analysis that is paramount to usability and productivity.

in our previous blog post – ‘the imperative for bioinformatics-as-a-service’ – we addressed the issue of the profusion of choice in computational solutions in the fields of bioinformatics research. traditionally, there has been a systemic, acute, and documented dearth of off-the-shelf technological solutions designed specifically for the scientific research community. in bioinformatics and omics research, this has translated into the necessity for users to invent their own system configurations, data pipelines, and workflows that best suit their research objectives. the output of this years-long diy movement has now generated a rich corpus of specialised bioinformatics tools and databases that are now available to the next generation of bioinformaticians to broker, adapt, and chain into a sequence of point solutions. on the one hand, next-generation high throughput sequencing technologies are churning out genomics data more quickly, accurately, and cost-effectively than ever before. on the other, the pronounced lack of next-generation high throughput sequence analysis technologies still requires researchers to build or broker their own computational solutions that are capable of coping with the volume and complexity of digital age genomics big data. as a result, bioinformatics workflows are becoming longer, toolchains have grown more complex, and the number of software tools, programming interfaces, and libraries that have to be integrated has multiplied. even as cloud-based frameworks like saas become the default software delivery model across every industry, bioinformatics and omics research remain stranded in this diy status. the industry urgently needs to shift to a cloud-based as-a-service paradigm that will enable more focused, efficient, and productive use of research talents for data-driven omics innovation and insights, instead of grappling with improvisation and implementation. how saas transforms bioinformatics analytics for the augmented bioinformatician even as the cloud has evolved into the de-facto platform for advanced analytics, the long-running theme of enabling self-service analytics for non-technical users and citizen data scientists has undergone a radical reinterpretation. for instance, predefined dashboards that support intuitive data manipulation and exploration have become a key differentiating factor for solutions in the marketplace. however, according to gartner’s top ten data and analytics technology trends for 2021, dashboards will have to be supplemented with more intelligent capabilities in order to extend analytical power – that thus far was only available to specialist data scientists and analysts –to non-technical augmented consumers. these augmented analytics solutions enable ai/ml-powered automation across the entire data science process – from data preparation to insight generation – and feature natural language interfaces for nlp/nlg technologies to simplify how augmented consumers query and consume their insights and democratize the development, management, and deployment of ai/ml models. specialized bioinformatics-as-a-service platforms need to adopt a similar development trajectory. the focus has to be on completely eliminating the tedium of wrangling with disparate technologies, tools, and interfaces, and empowering a new generation of augmented bioinformaticians to focus on their core research. enhanced scalability and accessibility a single human genome sequence contains about 200 gigabytes of data. as genome sequencing becomes more affordable, data from the human genome alone is expected to add up to over 40 exabytes by 2025. this is not a scale that a motley assortment of technologies and tools can accommodate. in comparison, bioinformatics-as-a-solution platforms are designed with these data volumes in mind. a robust and scalable saas platform is built to effortlessly handle the normalization, storage, analysis, cross-comparison, and presentation of petabytes of genomics data. for instance, our mindwalk platform utilises a container-based architecture to auto-scale seamlessly to handle over 200 petabytes of data with zero on-ramping issues. and scalability is not just about capacity. saas platforms also offer high vertical scalability in terms of services and features that researchers need to access. all mindwalk platform users have a simple “google-style” search bar access to 350 million sequences spanning 11 of the most popular publicly available databases, as well as to in-built tools for sequence analysis, multiple sequence alignment, and protein domain analysis. over and above all this, saas solutions no longer restrict research to the lab environment. researchers can now access powerful and comprehensive bioinformatics-as-a-service via laptops – or even their smartphones if mobile-first turns out to be the next big saas trend – in the comfort of their own homes or their favourite coffee shop. increased speed and accuracy bioinformatics has typically involved a trade-off between speed and accuracy. in some cases, methodologies make reductive assumptions about the data to deliver quicker results, while in others the error rate may increase proportionally to the complexity of a query. in multi-tool research environments, the end result is a discrete sum of the results received from each module in the sequence. this means that errors generated in one process are neither flagged nor addressed in subsequent stages, leading to an accumulation of errors in the final analysis. a truly integrated multi-level solution consolidates disparate stages of conventional bioinformatics and omics data analysis into one seamlessly integrated platform that facilitates in-depth data exploration, maximizes researchers’ view of their data, and accelerates time-to-insight without compromising on speed or accuracy. access to continuous innovation with a saas solution, end-users no longer need to worry about updates, patch management, and upgrades. with vertical saas solutions, such as bioinformatics-as-a-service, continuous innovation becomes a priority to sustain vertical growth in a narrow market. for users, this translates into more frequent rollouts of new features and capabilities based on user feedback to address real pain points in the industry. for instance, in just a few months since the official launch of our platform, we have added new capabilities for sdk/api-based integrations for proprietary data and infrastructure, expanded our tools and expertise to assay design, drug development, gene therapy, crop protection products, and biomarkers, and we are building out an ai platform with state-of-the-art graph-based data mining to discover and synthesise knowledge out of a multitude of information sources. the imperative to saasify bioinformatics saas is currently the largest segment in the public cloud services market – and yet the segment’s footprint in bioinformatics is virtually non-existent. today, there are a few cloud-based technologies targeted at genomic applications that focus on specific workflows like sequence alignment, short read mapping, snp identification, etc. however, what the industry really needs is a cloud-based end-to-end bioinformatics-as-a-service solution that abstracts all the technological complexity to deliver simple yet powerful tools for bioinformaticians and omics researchers.