Welcome Guest Search | Active Topics | Members | Log In | Register

v2.1.2 error importing "found files" CSV file Options
jmce_999
Posted: Monday, February 16, 2009 6:26:20 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Hi Oliver,

I tried the found files export/import feature to CSV. It did not work. I received a popup "ReplaceMagic - Error!: Import file format in not OK!"

The CSV export file is 1.63MB listing approx 5271 xls files. The CSV opens OK in Excel and format is clearly readable as:
<workbook name>;<# of Hyperlinks>;<# of OLE>;<folder path>;<file size>;<read only>

Reducing the row count (test was to reduce to 5 rows) of this file enabled the import function to work.

Looking forward to your feedback
Jim
Oliver
Posted: Monday, February 16, 2009 7:17:18 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Hi Jim,

can you send me this file for test?
You can use support @ replacemagic.com address.

Thanks,
Oliver

Oliver
ReplaceMagic Support
Oliver
Posted: Monday, February 16, 2009 7:22:46 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
I sent also email to you :-).

Oliver
ReplaceMagic Support
jmce_999
Posted: Monday, February 16, 2009 7:40:34 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
I'll send you a test file later today, will be in your inbox for tomorrow morning.
jmce_999
Posted: Monday, February 16, 2009 8:58:10 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Quote:
Hi Jim,

I guess that you manually created/extended list of found files. If this is a case then there is a problem (we are working on it).

I tested RM with folder where I have >1000 files and I did not have any problems to import that big list but when I manually
added one more file I had problems with import.

Anyway, we are working on this issue.

Best regards,
Oliver
ReplaceMagic team


No, I did not manually create the extended list, I allowed RM to search for XLS files, save the results then a few days later tried to import the file. I did not open the file at all prior to attempting the import.

You parse the "found files" file in reverse, from last file found to first? I noticed the BOM marker 0xFFFE prefixing the name of the last XLS file when I imported and checked a previously exported CSV containing 32 rows. Could there be other file delimiting characters causing an issue somewhere?

I'll check some more as to where the fail occurred.

As you are fixing some other bug can you fix something else I found? When I accidentally click CLOSE button (lower right of RM window) is it necessary to have to click CANCEL twelve times before the cancel request is acknowledged?
Oliver
Posted: Monday, February 16, 2009 9:31:47 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Thanks for info about close.
That will be fixed also.

Best regards,
Oliver

Oliver
ReplaceMagic Support
Oliver
Posted: Monday, February 16, 2009 10:06:45 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Can you check do you have duplicate files in your list?

This should not happen but currently we do not have check for that. Lists/Collections do not allow duplicates and this can be one reason why RM reports that import file is not ok.

On the other side I create one "Found Files" with >1000 files and it work ok.

If you can create one example file which will cause error message that will be great.

Btw. Close issue is fixed and will be published as soon as we close this issue.

Best regards,
Oliver

Oliver
ReplaceMagic Support
jmce_999
Posted: Monday, February 16, 2009 10:44:41 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Thanks for the feedback.

I can force the error (and popup) if I stuff in an additional CRLF sequence (0D00 0A00) at the end of the CSV. The CSV file contained only 1 line i.e. 1 XLS found. Could the method you use to parse the CSV be bombing out, posting the invalid file popup, if it encounters and misinterprets the character stream as invalid?

I have sent you two files, Good_Found Documents.csv & Bad_Found Documents.csv

I'll check the duplicate filename but I'd expect this to be handled by RM as it is possible the same XLS filename could exist in different folders.

Jim
Oliver
Posted: Monday, February 16, 2009 10:47:10 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Thanks for file.
Same file in different folder is not a problem. Key is built based on folder & file name.

I will check files later and let you know about results.

Oliver
ReplaceMagic Support
jmce_999
Posted: Monday, February 16, 2009 11:54:30 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
It is not caused by same filename issue you outline, but here's the weird thing:

1. take the large CSV file (5271 rows) that will not import and open it with Excel
2. sort ascending
3. save as CSV (all 5271 rows)
4. try to import into RM
5. imports successfully (all 5271 rows)
6. export same list once more using the "save list under..." feature
7. close and reopen RM
8. try to import same CSV into RM
9. import fails
goto 1

Can you send me a debug build so I can find out which record out of the 5271 causes the file import error popup to appear?
Oliver
Posted: Tuesday, February 17, 2009 7:20:10 AM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Hi,

I sent email with link to "debug" version. In case of error you will see line number where error happend and also error description.

Can you send me screenshot of this message?

Thanks,
Oliver

Oliver
ReplaceMagic Support
Oliver
Posted: Tuesday, February 17, 2009 7:33:59 AM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
I just uploaded new EXE (zipped) where we implemented correction for import of found files.

It worked with your "Bad_Found Documents.csv".

Can you try? File is on same location like link in previous email.

If you confirm that this is ok we will rebuild installation and officially publish it.

Best regards,
Oliver
ReplaceMagic team

P.S. Thanks for your help.

Oliver
ReplaceMagic Support
jmce_999
Posted: Tuesday, February 17, 2009 5:15:11 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Hi Oliver, I will test the debug version today. I also wanted to mention, during my test yesterday, where I found 5271 files, I also noticed another strange output. I ran the test again overnight to confirm. At the end of the search instead of the expected console output and popup message:

Processing accomplished at:
File(s) found:
Total size:
Processing time:

RM repeats the processing started message:

Processing started at:
Folder:
Filter: Microsoft Excel-Document
Subfolders included

I have sent you a screenshot of the console.
Oliver
Posted: Tuesday, February 17, 2009 8:00:20 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Strange. I will create folders with a lot of files and check it.
Btw. what I noticed in screenshot you started on 16.02.2009 at 9:31:57AM and process finished at same time. How fast message appeared after you started scan process?

Best regards,
Oliver

Oliver
ReplaceMagic Support
jmce_999
Posted: Tuesday, February 17, 2009 9:35:04 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Yeah, the start/stop popup messages in the console are in error.

The search process ran overnight, processing a large amount of files.

After the last file was found, the popup message was not the "Processing Accomplished" string we would expect but the same string used at the start of processing.
Oliver
Posted: Tuesday, February 17, 2009 10:14:34 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
When you switched back to "Find Documents" tab could you see list of all found documents?


Oliver
ReplaceMagic Support
Oliver
Posted: Tuesday, February 17, 2009 10:19:48 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
Can you start again scan process and wait couple of minutes to see will found files appear in "Console" and "Find Documents" tabs?

Thanks,
Oliver

Oliver
ReplaceMagic Support
jmce_999
Posted: Tuesday, February 17, 2009 11:42:05 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Debug progress taken offline. Email on its way.
Oliver
Posted: Wednesday, February 18, 2009 9:56:03 PM
Rank: Administration
Groups: Administration

Joined: 12/5/2006
Posts: 98
Points: 894
As you found problem is with comma and semi-colon in file name.
New version of ReplaceMagic will replace , and ; with some other characters and later when you import found files replace them back.

Best regards,
Oliver

Oliver
ReplaceMagic Support
jmce_999
Posted: Monday, February 23, 2009 8:50:49 PM
Rank: Advanced Member
Groups: Member

Joined: 1/30/2008
Posts: 57
Points: 171
Location: Canada
Oliver wrote:
As you found problem is with comma and semi-colon in file name.
New version of ReplaceMagic will replace , and ; with some other characters and later when you import found files replace them back.

Best regards,
Oliver


Thanks, will now test latest release version.
Users browsing this topic
Guest


Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Main Forum RSS : RSS

FlatEarth Theme Created by Jaben Cargman (Tiny Gecko)
Powered by Yet Another Forum.net version 1.9.1.1 (NET v2.0) - 9/10/2007
Copyright © 2003-2006 Yet Another Forum.net. All rights reserved.
This page was generated in 0.133 seconds.