Skip to content
Infoveave Learning

Infoveave Datasources

Datasources are the backbone for accessing and managing different types of data. They act as dedicated storage spaces, forming the foundation for comprehensive data analytics and supporting insightful analysis and decision-making. Datasources in Infoveave cover various data formats like databases, files, and cloud services. Creating a Datasource is about telling where to find specific data, while configuring with specifying how this data should be organized and used. It is like giving Infoveave a map to locate and understand the available information. Datasources equip Infoveave to access and interpret data, making it a powerful tool for managing and analyzing diverse datasets.

Overview of Datasources

In the realm of data management, Datasources are broadly categorized into three types: structured, semi-structured, and unstructured. Each category encompasses diverse formats. The categorization of Datasources is based on the level of organization and rigidity in the format of the data they contain.

datasource-classification

Structured Datasources

Structured datasources refer to organized and well-defined data formats where information is neatly arranged in rows and columns. This type of data adheres to a predefined schema, making it easy to query, analyze, and process. Common examples include relational databases like MySQL, Oracle, or Microsoft SQL, where data is stored in tables with fixed column types and relationships. The structure allows for efficient storage, retrieval, and manipulation of data. Infoveave supports following structured Databases:

  • Apache Cassandra: Distributed and decentralized storage.
  • MariaDB: Open-source variant of MySQL.
  • Microsoft SQL: Versatile relational database.
  • MySQL: Renowned relational database by Oracle.
  • Oracle: Enterprise-level database.
  • Percona: Variant of the MySQL database.
  • Postgres: Open-source database.

Semi-Structured Datasources

Semi-structured datasources exhibit a more flexible organization compared to structured data. While there might be a loose organization, the data doesn’t strictly adhere to a rigid schema. Instead, it often includes tags, markers, or other structural elements that provide some level of organization. Examples include delimited files (CSV, TSV, PSV) where data is separated by specific characters, fixed-length files where columns have defined positions, and JSON (JavaScript Object Notation) files that allow for nested and flexible structures.

Unstructured Datasources

An unstructured datasource refers to data that lacks a predefined and organized format. This type of data doesn’t neatly fit into traditional databases or tables. Unstructured datasources can include text documents, images, videos, or other files with no clear schema. Examples of unstructured datasources in a general context include documents in various file formats (e.g., PDF, Word), multimedia files, and data from sources like social media or web scraping. Analyzing unstructured data often requires advanced processing techniques due to its flexibility and variability.

Infoveave Datasource Classification

Infoveave-Datasources

File Based Datasources

File-based Datasources in Infoveave provide a versatile way to connect and work with various file formats, enhancing your data management capabilities. It covers different file types and structures to suit your needs:

  • Delimited: Suitable for CSV, TSV, PSV, and custom delimited files, allowing flexible data separation and analysis.
  • Excel: Supports both XLS and XLSX formats, enabling direct interaction with Microsoft Excel files.
  • Fixed Length: Efficiently separates data into columns using positional information, helpful for structured data.
  • JSON: Connects to JSON-based APIs and automations, offering seamless integration with JSON data sources.
  • Multi File: Allows you to combine multiple file types within a single Datasource, simplifying complex data setups.
  • Plugin Parser: Enables the upload of custom C# plugins for parsing specific file formats, enhancing data interpretation.

Databases

Databases in Infoveave provide a solid foundation for storing and managing your data efficiently. Here are some of the key database options available:

  • Apache Cassandra: Offering decentralized storage, Cassandra ensures reliable and scalable data management.
  • MariaDB: An open-source version of MySQL, MariaDB is a powerful choice for enterprise-level database needs.
  • Microsoft SQL: Microsoft’s relational database solution supports both on-premises and cloud environments.
  • MongoDB: A NoSQL database, MongoDB is designed for flexible and dynamic data handling.
  • MySQL: Oracle’s relational database management system is widely used for structured data management.
  • Oracle: An enterprise-grade database, Oracle is available for both on-premises and cloud deployments.
  • Percona: A variant of MySQL, Percona offers enhanced features for MySQL-based data management.
  • Postgres: An open-source database, Postgres is versatile and adaptable for both on-premises and cloud usage.

 Cloud Services

Infoveave offers seamless integration with various powerful tools to enhance your data analysis capabilities. Here’s a glimpse of the cloud services available:

  • Amazon Athena: A serverless solution by Amazon for interactive analytics, Athena enables you to query data effortlessly.
  • Amazon Redshift: Managed by Amazon, Redshift provides a scalable and high-performance data warehouse service suitable for large datasets.
  • Data World: With Data World, you can manage and explore your data catalog effectively, catering to the modern data landscape.
  • Druid: Druid is designed for real-time analytics, allowing you to swiftly analyze and visualize data in dynamic ways.
  • Google Analytics: Infoveave seamlessly connects to Google Analytics properties, supporting both V3 and V4 versions, enabling you to gain insights from web analytics data.

Others

Infoveave offers diverse data integration options beyond the conventional, providing you with flexibility and adaptability. Explore these unique Datasource types:

  • Custom File Parser: Take control of parsing unstructured text files with custom code. This option empowers you to tailor data extraction to match your specific file formats.
  • IoT: Create IoT Datasource that listens to devices, capturing real-time data streams. This dynamic approach enables you to harness insights from the world of connected devices.

Connecting External Systems to Infoveave

Let’s delve into the fundamental concept of connecting external systems to Infoveave, shedding light on its necessity and the diverse types of connections that empower you with data integration.

  • Understanding the Essence: Connecting external systems to Infoveave is the gateway to a richer, more expansive data landscape. It involves establishing links with external sources, be it databases, cloud services, or even data from the Internet of Things (IoT). This process allows Infoveave to tap into a myriad of data streams, creating a unified ecosystem for comprehensive analysis.
  • The Necessity of Connection: Why is connecting external systems so crucial? The answer lies in the diversity of data. External systems house a treasure trove of information that, when seamlessly integrated into Infoveave, enriches the analytical capabilities. From customer databases to real-time IoT sensor data, the necessity lies in unlocking the full potential of varied data sources for robust decision-making.
  • Types of Connections: The diverse types of connections that Infoveave facilitates includes traditional databases like MySQL and Oracle, cloud services such as Amazon Athena and Redshift, and even data streams from IoT systems. Each connection type brings its unique advantages, offering users the flexibility to merge structured and unstructured data seamlessly.
  • Data from IoT Systems: Integrating data from IoT systems involves connecting Datasources to sensors, devices, and IoT for real-time data analysis, offering insights into evolving datasets. This is especially useful for industries that require continuous monitoring and quick decision-making.

Conclusion

In conclusion, Datasources are the backbone of Infoveave, enabling efficient data analytics across various formats like databases, files, and cloud services. The categorization into structured, semi-structured, and unstructured types provides a comprehensive framework for data organization. Additionally, Infoveave’s diverse Datasource classifications, including file-based, databases, and cloud services, offer versatile options for efficient Datasource creation, management and integration.