A high volume of data is gathered by applications in various industries at a very high rate. The collection of data from various sources is a complex task. The data is not always available from standard database stream. They could be file system, database, feeds, through web services, interaction with legacy systems, etc., In this section we will discuss about various data operations that occur in collecting data from various sources.
Handling data is one of the most complicated and key functionality of an application. However one of the key challenges faced is the format of the data. There are legacy data, data available from various data providers, etc., ETL - Extract, Transform and Load promises to resolve the heterogeneous data format. In this research project I analyze the ETL process and present a solution for continuous integration of heterogeneous data in an application.
The information and services available to the applications extend beyond classical database and file system methods. Other methods through which data available are, for example, Web Syndication, Web services, Web API, Legacy Application Interaction etc. These contribute to the enormous volume of data and numerous formats an application needs to handle. To facilitate the process, we present a framework solution that will enable applications to consume data in various format and make it homogeneous format.
No comments:
Post a Comment