IT Process Automation

Expand all | Collapse all

Read_from_file Operator. How to read more than 1024 lines?

Jump to Best Answer
  • 1.  Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-07-2017 04:15 PM

    Hi Community,

     

    I am facing a problem with reading txt files. The "read_from_file operator" only reads 1024 rows and I have files bigger than that. My idea is to check the number of rows in a file and run the loopback operator a few times by adding the result to a process variable. Is there an operator that tells the number of rows in a file?

    Any other way to read a file with more than 1024 lines?

     

    Thank's in advance. 



  • 2.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-07-2017 04:51 PM

    The read from file operator allows you to specify the the line number to start and stop reading.  You would need to read the additional lines from a new operator and write to a new variable in Process Automation.

     

    I don't see a built in way to get the row number out of a file using the out-of-the-box PAM operators.  The "Get File Attributes" operator returns things like size, but I don't see a row count.  You might have to build a java function to do that.



  • 3.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-09-2017 11:31 AM

    Hi Andrew,

     

    I realized that I would have to build something using some programming language. But before that i wanted to know if anyone in the community had this and the ideas used. I've created a solution that I found easy to implement and maintain. I will show my solution in this same post.

     

    Thank you for your help.



  • 4.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-09-2017 10:40 AM

    Hi Mario, I just run a few tests with this operator and the latest pam version (4.3.02) and I was able to get an datase of 1867 lines (it's a txt file).

     

    Did you test just unchecking the "Return file contents in a string array" and split the output by "\n"?



  • 5.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-09-2017 11:35 AM

    Hi Deama10,

     

    This solution is very good, but I think it would be limited by the size of the file, but this solution should work for simpler cases. In my case it was important to bring information in array form for easier use ... I've created a solution and I'll show it here.

     

    Thank you



  • 6.  Re: Read_from_file Operator. How to read more than 1024 lines?
    Best Answer

    Posted 08-09-2017 11:50 AM

    Hello guys,

     

    My solution to this case was to use an MS-DOS function to count the rows of the file. I used the FIND function to count the rows, since my file has the csv format, so I searched the lines that had "," and used that value to run the operator as many times as needed. The result of the load I stored in a process variable. Here are the operators and code I used.

     

     

    MS-DOS function:

     

    find /C /I "," file_name.txt

     

    Code to set start and end limits:


    if (Process[OpName].CurrentLoopIteration == 0) {
    Process.Inicio01 = 2;
    Process.Fim01 = 1023;
    }
    else {
    Process.Inicio01 = Process[OpName].CurrentLoopIteration * 1024;
    Process.Fim01 = Process.Inicio01 + 1023;
    }

     

    Please, feel free to comment on the solution.

     

    Thank you all.



  • 7.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-09-2017 01:41 PM

    This on post execution would also return an array.

    Process.linhas = [];
    Process.linhas  = Process[OpName].DatasetVariable.split('\n');

     

    I wasn't able to reproduce the cap of 1024 rows on the latest process automation version. 



  • 8.  Re: Read_from_file Operator. How to read more than 1024 lines?

    Posted 08-09-2017 02:50 PM

    Marcel,

     

    I'm using CA PAM 4.3 SP1. 

     

     

    Did CA change this limit in the newer version? Is there any documentation of this?

     

    Thanks