Re: Splitting huge XML Files into fixsized wellformed parts

On 18 Mrz., 00:01, Hermann Peifer <pei...@[EMAIL PROTECTED]
> wrote:
> Malapha wrote:
>
> > As I come from the VBA world - I tried to get familiar with awk. What
> > I do have is theoretical solution in form of a structured process
> > diagram :-)
>
> > Copy Header and Footer from Original to Var
> > Set Start_Offer =3D First Offer (from <Offer> to </Offer>)
> > Set End_Transaction =3D 0
> > Set Part =3D 0
> > Set FileSize =3D 0
> > Set MaxFileSize =3D 250
> > while not Start_Offer < EOF(OriginalXMLFile)
> > =A0 =A0 =A0Part=3Dpart+1
> > =A0 =A0 =A0Open NewFile OriginalXMLFileName + Part + ".xml"
> > =A0 =A0 =A0Paste Header from Var to NewFile
> > =A0 =A0 =A0While filesize(NewFile)<MaxFileSize do
> > =A0 =A0 =A0 =A0 =A0Copy Offer (Start_Offer) from OriginalXMLDatei to
New=
File
> > =A0 =A0 =A0 =A0 =A0Start_Offer=3DStart_Offer + 1
> > =A0 =A0 =A0wend
> > =A0 =A0 =A0Paste Footer from Var to NewFile
> > wend
>
> > I am right now trying to translate this into awk.. Please dont ask me
> > how far i am, its frustrating :-)
>
> Below one solution for splitting in well-formed chunks, here: 100
> OfferInfos each. =A0There might be better solutions (I just don't know
> them ;-) It only works if the XML data is in "pretty print format", as
> the sample data you posted.
>
> $ cat split_bigfile.awk
>
> BEGIN { new_chunk =3D 1 ; size =3D 100 }
>
> NR =3D=3D 1 { header =3D $0 ; next }
> NR =3D=3D 2 { header =3D header ORS $0 ; footer =3D "</" substr($1,2)
">" =
; next }
>
> $0 !~ footer {
> =A0 =A0 =A0 =A0 if (new_chunk) {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 outfile =3D "chunk" sprintf("%07d", num)
"=
..xml"
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 print header > outfile
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 new_chunk =3D 0
> =A0 =A0 =A0 =A0 }
> =A0 =A0 =A0 =A0 print > outfile
>
> }
>
> /<\/OfferInfo>/ {
> =A0 =A0 =A0 =A0 num =3D int(count++/size)
> =A0 =A0 =A0 =A0 if (num > prev_num) {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 print footer > outfile
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 new_chunk =3D 1
> =A0 =A0 =A0 =A0 }
> =A0 =A0 =A0 =A0 prev_num =3D num
>
> }
>
> END { if (!new_chunk) print footer > outfile }

Herman you are great. As I have written in to J=FCrgen, I am unable to
check it. But as soon as possible I ll give it a try!!

Thanks again
Malapha