Home All Groups Group Topic Archive Search About

Where exactly the data is stored when we create DataTable

Author
16 Jun 2009 5:00 AM
Neha
Hi,

I want to go for datatable instead of arrays. As I want to keep large
amt of data.
I just want to know. If I am exceeding the amount then will that be
saved in disc. Will it take care of memory issue.

Otherwise I will have to go for arrays and go on writing the same to
disc.

Please let me asap.

Thanks,
Neha

Author
16 Jun 2009 5:28 AM
Alberto Poblacion
"Neha" <nehab.i***@gmail.com> wrote in message
news:e0bb9b2a-a6aa-48a3-a419-3ef4f61414d8@z8g2000prd.googlegroups.com...
> I want to go for datatable instead of arrays. As I want to keep large
> amt of data.
> I just want to know. If I am exceeding the amount then will that be
> saved in disc. Will it take care of memory issue.
>
> Otherwise I will have to go for arrays and go on writing the same to
> disc.


    A DataTable will be allocated exactly the same as an Array: it will use
memory from the Heap, which is in turn allocated from virtual memory. This
means that it will initially reside in RAM but parts of it may be paged into
disk as needed by the Operating System.

    If you need to manage an amount of data that is too large to be kept in
memory, you may wish to consider using a database. ADO.NET contains classes
that will let you move (parts of) a table stored in a database from/to a
DataTable in memory.
Are all your drivers up to date? click for free checkup

Author
16 Jun 2009 5:39 AM
Peter Duniho
On Mon, 15 Jun 2009 22:00:46 -0700, Neha <nehab.i***@gmail.com> wrote:

> Hi,
>
> I want to go for datatable instead of arrays. As I want to keep large
> amt of data.
> I just want to know. If I am exceeding the amount then will that be
> saved in disc. Will it take care of memory issue.
>
> Otherwise I will have to go for arrays and go on writing the same to
> disc. [...]

All else being equal, a DataTable will have more overhead to store the 
same data than an array will.  And a plain in-memory DataTable will not 
persist the data automatically, though of course for both the plain 
in-memory DataTable and an array, if the size of your data exceeds the 
physical RAM available, Windows will automatically use the disk as 
temporary storage via the virtual memory management system.

It seems to me that one main advantage to using the DataTable is that it 
includes support, via a DataAdapter, for connection to an actual database 
somewhere.  If you can benefit from that feature, then you will probably 
want to use the DataTable in spite of the additional overhead.  Similarly, 
the DataTable provides a higher-level view of the data, even for a plain 
in-memory one, and that abstraction may make the rest of your code 
simpler.  That would be another reason to use the DataTable class in spite 
of the additional overhead.

Otherwise, if you're using the DataTable as nothing more than an array, 
you might as well just use an actual array; there wouldn't be any 
memory-management advantage to using a DataTable over an array.

Pete

Bookmark and Share