Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I am getting flat file parse exception when i use FlatFileItemReader in my job configuration.

Actually it was working fine, but with recent files its throwing error.

org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 1 in resource=[URL [file:/prod/users/cdi/crh537/Java_All/input/xaaaaa]], input=[16910203315393417SEVAN Q DANG18711 PARK GROVE LNDALLASTX7528751224609645044438000VANVETTE98@YAHOO.COM586404562MAR CL197311192013042504Closed (zero balance DDA)220PUSA]
The main difference is current file records contains some special characters like @ and ().
The following line throws the error. The delimiter used is ^A (linux)
16^A123^A2323232^ADD^SWAN D REE^A123 MMM STRRET^A^ADALLAS^ATX^A2222^A^A^A^A^A^A^A^A^A^A434343434^A23232^A^AABC@ABC.COM^A586404562^A^A^AMAR^A ^ACL^A121212^A20130425^A04^AClose^A^A220^A^A^AP^AUSA^A
The following is the reader configuration.
<bean id="masterFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
        <property name="resource" value="#{stepExecutionContext['fileResource']}" />
        <property name="lineMapper">
            <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <property name="lineTokenizer">
                        class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                        <property name="delimiter" value="${file.delimiter}"/>
                        <property name="names" value="filed1,field2,...field39" />
                    </bean>
                </property>
                <property name="fieldSetMapper">
                    <bean class="com.batch.SrcMasterFieldSetMapper" />
                </property>
            </bean>
        </property>
    </bean>
                Actually the issue was the record ending with ^A$ so, its considering one extra token(expected 39 , actual 40).. Thanks for the detail stack trace which i somehow missed in my code.
– Shankar
                Mar 12, 2015 at 7:03
I guess you have hidden characters in your file. You can follow this link instructions to verify if and which characters you have in your file.
This question can help you \001 delimiter issues regarding choosing right value for delimiter. As suggested you can try to pass "\u0001" instead ^A.
Update:
The actual error was when printed the entire stack trace org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 39 actual 40
Its because the last field in a record/line followed by delimiter and $ not just $, so there is one extra token.
                Thanks @Nenad.. I will look into the links and let you know. BTW i am passing "\u0001" as delimiter in my code.
– Shankar
                Mar 11, 2015 at 14:54
                Maybe you can also put whole stack trace since FlatFileItemReader is wrapping exception from LineTokenizer and it holds actual error which might help in resolving issue.
– Nenad Bozic
                Mar 11, 2015 at 15:17
                Probably exception from LineTokenizer held exception that it expected 39 tokens but had 40, maybe you might add it to your post so when somebody finds this posts they will have easier time knowing what was wrong.
– Nenad Bozic
                Mar 12, 2015 at 8:05
                Yes @Nenad... when i print the whole stack trace the actual error was org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 39 actual 40.... But FlatFileItemReader wrapping that exception and gives some commong parsing exception.
– Shankar
                Mar 12, 2015 at 8:56
                Yeah I know but might be good for future readers of this question to edit your answer and add that exception, was just suggestion. Not many people read this comments.
– Nenad Bozic
                Mar 12, 2015 at 8:58
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.