Extracting multiple fragments from array objects

[Ray]

Banging my head against wall all day!

I'm reading an ASCII file and collecting the file lines into an array. Each line has some data I want to extract, in exactly the same column on each line. There is no consistent delimiter, so I can't split each line into reliable fragments.

An example of a line:

 | 31  9 |  0.9  4   |    0.8 11.5  46

So I'm trying to slice the fragments I need, e.g:

line[3,2]

Trouble is I can't seem to grab more than one fragment from a line at a time. I wan to be able to do (for each line):

puts(line[3,2] + "" + line[6,2]..)

Is there an easier way?

[zeroedin]

I'm not a ruby person, but can't you string replace the spaces then explode the whole lot?

[Ray]

Cheers zeroedin. I'm new to Ruby too.

Ya that's more or less what I ended up doing. I think the line[] slice was working, but getting caught on blank spaces.

I replaced all pipes, spaces and some other noise with '0'. Now I think I can reliably extract values from specified positions on each line.

def clean
  @data = IO.readlines("path/to/rawfile.txt")[9..20].collect { |line| line.gsub(/["| *"]/, '0')}
  File.open("path/to/cleanfile.txt","w") do |out|
    out.puts @data
  end
end

[temp]

Your initial approach should work fine.

ruby-1.9.3-p0 :007 > line = "hello     |   foo     |  &  X 0    bar"
 => "hello     |   foo     |  &  X 0    bar" 
ruby-1.9.3-p0 :016 > line[0,5] + line[14,3] + line[35,3]
 => "hellofoobar"

With your second approach, you need to be careful that your special delimiter character (0), doesn't appear in the data (which, in your example data, it does).

[nebbian]

Can't you just use the Ruby equivalent of substring?

Here's how I solved a similar problem using Java...

The data parsed looks very similar to what you've got. Here are some sample lines:


!@[139|7687=L|003|003|22 |J.Brock | |002|00:01:41.5815| |002|00:01:41.5815|000|00:00:01.2505|000|00:00:00. 4604|0000|0000|0000|0000
!@[139|7931=L|004|004|46 |J.Harrigan | |001|00:01:49.6920| |001|00:01:49.6920|000|00:00:09.3610|000|00:00:08. 1105|0000|0000|0000|0000
!@[139|8033=L|005|005|71 |M.Zukanovic | |001|00:01:51.5263| |001|00:01:51.5263|000|00:00:11.1953|000|00:00:01. 8343|0000|0000|0000|0000


In my application each line has a header that says what type of line it was. Once I'd read the header, I'd load up a class that parsed that line. Each parsing function looked something like this:

	public void parseString(String inputString) {
		String 	tempBuffer;
		int		i;
		
		gDataValid = false;

		if(inputString.length() >= (recordSize - 2)) {
			gHeader.parseString(inputString);

			gIndicator = inputString.substring( indicatorStart,indicatorStart + indicatorLength);

			tempBuffer = inputString.substring( lineNumberStart,lineNumberStart + lineNumberLength);
			gLineNumber = HelperFunctions.safeParseInt(tempBuffer);

			gPosition = inputString.substring( positionStart,positionStart + positionLength);

			gCarNumber = inputString.substring( carNumberStart,carNumberStart + carNumberLength);

			gDriverName = inputString.substring( driverNameStart,driverNameStart + driverNameLength);

... etc.

I had a big list of variables in each class that encoded the start position and length of each record in the data stream.

Then when I needed a parsed variable from that line I'd ask that class to give it to me.


This system worked flawlessly for years, and as far as I know is still in use by a couple of clients.


Hope this helps :-)

[Ray]

Cheers for your help guys!

@Nebbian - is that the F1 pole positioning app?

@temp - you're right, my original approach did work fine in the end. I just can't work out why it wasn't working the first time.

This is what I ended up with:

def clean
  File.open("path/to/cleanfile.txt","w") do |out|
    data = IO.readlines("path/to/rawfile.txt")[7..195].collect { |line| line[3,2] + "," + line[6,2] + "," + line[12,3] + "," + line[30,4] + "," + line[36,2] }
    out.puts data
  end
end

Then followed that up with a method to import the CSV to MySQL. It works!

[nebbian]

Originally Posted by Ray

Cheers for your help guys!

@Nebbian - is that the F1 pole positioning app?

Close -- it's the application that several V8 Supercar teams run in their pit garage when they want to see how their cars are doing on the track.


Glad you got your code working in the end :-)