Identifying the data needs of an application is critical to designing a data solution. Data sets are growing , range of data types are widening and data manipulation is more demanding. All highlighting the requirement to understand the application data requirements.
In theory, an Architect or DBA will have an opportunity to ask the question “What are the data requirements?”. Gathering facts about the data type range, demand for the data , such as 24 x 7 and workload allows the DBA to design a solution maximising the return – financial and performance – on storage , manipulation and on-going management. There is cosntant pressure to achieve Database cost savings
You may not have noticed but up to this point I haven’t used the word “database”. This is important – as I haven’t arrived with any preconceived ideas about a data solution. Data requirements have changed significantly over the past 20 years in terms of scale and manipulation requirements. Resulting in development of new design and data growth patterns.
Relational versus NoSQL is a stark example of a new design pattern. Relational is very firmly underpinned by ACID , critical to data integrity. NoSQL is a schema – less in nature and serves a different purpose.
There has been much debate about the future of relational databases. NoSQL, big data, MongoDB,Hadoop , sql v nosql.. all terms in the media and all packed with potential. It’s only when you start working these technologies you start to realise the strengths and weaknesses.
A good data modeller can take the most complex data structures and integrate into a relational database and then create a query framework to extract the data. But why? When a more elegant solution may be to use a nosql solution .
These are sorts of decisions DBAs and Data Architects need to make. It comes back to understanding the data needs and proposing a solution relevant to the problem. RDBMS remains the core for most organisations , as its influence spreads across every aspect of the organisation, and to the developer offers transaction management, but non-relational patterns offer alternative ideas