Very Big Genetic Database

[dcabe]

We are working on Oracle 11G
It seems that the relational model reach its limit when we have to store Genetic Datas (SNiP results)
the table we have to store these kind of Datas look like:
the key is bases on 2 fields:
-Id sample (numeric key)
-Id molecular marker (numeric key)
the result we have to store is simply a short character field as letter or combination of lettres: exple: T or A/T or G or C

the specificity is that, by experiment ,
we have a lot samples (2000 ... 10000)
and very great num markers (30000, 50000, -> 200000)
so the file we have to import in our tables is a text matrix
mk1 mk2 mk3 ....... mk20000
sample1 A T A/T
sample2 G G/T A
...
Finally , a text file no so big... but when we convert these datas in relational model, we have a very very large number of rows (exple: 5000 samples * 200000 mk)
The Key of the table is bigger than the info stored...
the problem after is that we need to search results by list of sample and /or big list of markers... So the two id (sample and mk) must be used...
I 've started to search infos concerning Nosql databases and Big data database... But I have no really answers to our problem to store this type of Data.
Could you help me ? or give me some ideas to progress ?
Many thanks in advance
D. CABERO

[Tim...]

Hi.

Well, it does sound like the relational model may not be ideal for what you are attempting. If your search requirements are relatively simple, it sounds like you may get a better result from using one of the noSQL databases. There are a number of them around, including one from Oracle.

Typically people use these for processing large amounts of name:value pair style data. They are pretty good at coping with high volume, simple data, provided your querying is not complicated also. Where they break down is when your queries against the data are complex. I think it is worth doing a proof of concept with it, if only to discount it if it is not the way to go for you.

Cheers

Tim...