|
ms
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Process File in Chunks - StreamReadergoes! Hi All, I want to process a file in chunks, meaning I have a big file of 50K rows. I want to process 5K rows at a time. I am using StreamReader object and looping through each row and process it, and as I process it I create an XML file and send it to DB to be processed. while(oSR.Peek() > -1) { // 1. Create Xml from these lines } // 2. Call DB Functions. How do I read only certain number of lines at a time, complete my steps 1 and 2 and come back and read another 5K lines and so on.... any ideas gurus? thanks This is kind of hokey, and untested, but should give the idea:
StringBuilder sb=null; while(oSR.Peek() > -1) { sb=new StringBuilder(); for(int i=0;i<5000;i++) { if(oSr.Peek()>0-1) sb.Append(oSR.ReadLine()); } ProcessMyXml(sb.ToString();) } -- where ProcessMyXml accepts the string containing the 5000 lines, you can Split ('\r') on the newline to get into a string[] array for processing. and Split again on each row if it has delimited info in it. Peter -- Show quoteHide quoteCo-founder, Eggheadcafe.com developer portal: http://www.eggheadcafe.com UnBlog: http://petesbloggerama.blogspot.com "das" wrote: > I posted this on ado.net, which seems like the wrong group - so here it > goes! > > Hi All, > I want to process a file in chunks, meaning I have a big file of > 50K rows. I want to process 5K rows at a time. > > I am using StreamReader object and looping through each row and process > > it, and as I process it I create an XML file and send it to DB to be > processed. > > > while(oSR.Peek() > -1) > { > // 1. Create Xml from these lines > > > } > > > // 2. Call DB Functions. > > How do I read only certain number of lines at a time, complete my steps > > 1 and 2 and come back and read another 5K lines and so on.... > > > any ideas gurus? > thanks > > 
Other interesting topics
Having trouble with multiple threads and a shared resource
How the devil do I get screen resolutions??!!? Another form question Reads XML node from App.Config and populates a Class Using properties from a nested class trivial question about building exe Debugging a command line c# windows Application Enterprise Library Application Blocks question... picture compare Help Passing TreeView to another form and then back |
|||||||||||||||||||||||
