plainblack.com
Username Password
search
Bookmark and Share

    

Import script for collaboration systems

User Trex
Date 8/28/2010 10:58 am
Views 6137
Rating 22    Rate [
|
]
Previous · Next
User Message
Trex

I would like to import a series of blog posts (product reviews, actually) into a Collaboration System from another CMS-like system. Has anyone out there done this sort of thing before and have an import script they would like to share with me to speed the process along?

I was able to find one forum discussion from 2007 that covers this topic, but it only discusses the idea in theory and doesn't give examples: http://www.webgui.org/etcetera/collaboration-system-import-and-export

There is also one USS import utility from 2005 in the Bazzar, but I think that API is a bit too old in the tooth for my needs: http://www.webgui.org/bazaar/uss-import-script

I also installed wgd to have a look at what it can do, but its import tools appear limited to WebGUI's import/export files.

What I would like to be able to do is import via CSV/tab-delimited file, somewhat like how the user import script works, only mapping the columns to basic CS fields (including the userDefined fields). Fortunately, I do not need to bother with importing images or other file attachments, just standard title, description and userDefined fields.

The userImport utility is nice in giving examples of how to deal with CSV input, but beyond that, it deals with importing users, not assets. The galleryImport and fileImport utilities are handy for showing examples of importing assets and adding them as children of some specified asset, but they are complicated by including a lot of stuff I don't need regarding the creation of storage space and adding files to it.

If anyone has created an import script for their own use that is closer to my needs than the above examples, I would be very grateful to see it! I'm sure I could create one on my own with enough time, but with all the other things I have to do, finding starting point on this project that is closer to my own needs would be very helpful.



Back to Top
Rate [
|
]
 
 
perlDreamer

Several of the WebGUI tests create threads and posts to go into a Collaboration System.  They'll show you enough of the API that you can combine them with a script for parsing something on the front end, and then create the CS on the backend.

Try t/Asset/Post/trashing.t, for example.



Back to Top
Rate [
|
]
 
 
Trex

Thanks for the pointer, Colin! Testing is one of the fields in WebGUI that I have not yet tackled, so I didn't think to look there. I'll have to take a look...sometime after the WUC.



Back to Top
Rate [
|
]
 
 
martien

What I would like to be able to do is import via CSV/tab-delimited file, somewhat like how the user import script works, only mapping the columns to basic CS fields (including the userDefined fields). Fortunately, I do not need to bother with importing images or other file attachments, just standard title, description and userDefined fields.

I could help you with a script that does this from a directory with files and a csv structure like:

"moduleshort;moduletitle (chapter);foilname;foiltext;"

My app is/was converting  ppt->pdf->images + csv.

It should be expanded to use it from a client-system.

 



Back to Top
Rate [
|
]
 
 
Trex

I could help you with a script that does this from a directory with files and a csv structure like:

"moduleshort;moduletitle (chapter);foilname;foiltext;"

My app is/was converting  ppt->pdf->images + csv.

It should be expanded to use it from a client-system.

This sounds like it could indeed be helpful! Thank you. However, I'm not sure how the labels in your example correspond to WebGUI's.

Fortunately for this project, my needs are relatively simple: I don't need to include comments/replies, so the process would simply add a series of threads to a specified collaboration system. As such, the script would take as parameters:

  • collaboration system ID
    (you would manually set up the CS before running the script)
  • username
    (the person under whose name the posts will be posted, which could be overridden in the CSV file)

Then, for the input file, you would have a CSV where the first row defined the standard post fields, and each row thereafter included data that corresponded to the post fields -- essentially, the data form elements available in the post template:

  • title
  • content
  • username
  • dateSubmitted
  • views
  • userDefined1-5

Aside from the obviously required fields (title & content), the rest would be optional, though you probably need a username either here or as a global parameter passed to the script. I presume that the CSV file would have to be in chronological order (oldest posts first, which should correspond to dateSubmited) in order for them to be added in proper order to WebGUI.

If you are interested in discussing further off the list, please feel free to send me a private message.



Back to Top
Rate [
|
]
 
 
Trex

I have created an import script, and it appears to be functional all except for one problem: each post that is added gets set up with its own version tag separate from the tag set by the import script's start subroutine, and these tags are not committed. I have attacted my current working version of the import script to this message.

The script takes as parameters either the ID or URL of the parent asset under which the children will be appended, a CSV file (with header row) of properties for the child assets, and class name (in case you want to use this for assets other than posts. It sets the parent asset with WebGUI::Asset->newByUrl or WebGUI::Asset->new, accordingly, then appends children using WebGUI::Asset->addChild. This is all within the start and finish subroutines of the utility skeleton, which include setting a version tag. (Yes, this is very much a working script, with non-standard documentation and error checking at this point.)

The script started with the basic _utility.skeleton, added file reading from userImport.pl, but added Text::CSV_XS for better handling of quoted strings and escape characters, then built and appended children using fileImport.pl and galleryImport.pl as examples.



Attached Files
Back to Top
Rate [
|
]
 
 
Trex

Many thanks to Haarg on IRC for finding I had %options = {} rather than = (). That was the problem.

I'm considering cleaning this up, adding some mods (making it a general asset importer, not just for CS, and allowing for threaded lineages), then putting it up on the bazaar. I'd be happy to hear feedback -- would other people find it useful and any suggestions on the code itself.



Back to Top
Rate [
|
]
 
 
    



© 2019 Plain Black Corporation | All Rights Reserved