Converting DB data from Xenzine to Showcase

[kram242]

We are currently running an implementation of Xenzine (no longer regularly updated) and hoping to migrate data to Showcase (newer, regularly updated, prettier and better posting for users) .

So far, i've setup a Lab (Clean install of XF1.5.17), and installed Showcase onto the lab as well.

As a first test, I wanted to see how easy it would be to populate Showcase directly from the DB side (Would be easier to manipulate the Xenzine data, and export into the DB)

1) I used the frontend to create a new showcase item
2) From mysql, I copied the row in xf_nflj_showcase_item
3) Got really excited when I refreshed the frontend, and the copied item showed up in the list, below the original item
4) Clicked on the copied item, couple issues
4.1) Images that was inserted before, is just shown as "View attachment xx" links.
4.2) Cover image for the copied item isnt shown
5) Going back to Showcase after viewing the item, item disappeared from view
6.1) Checked DB, row 2 is still there
6.2) Found item under Showcase -> Your Items - both are shown there (original and copied row)
7) Comparing the copied row to the original row doesnt yield any immediate sign as to why it went invisible. It's item_state is still visible. Only thing i notice is cover_image_id is set to 0 everytime i view the item
8) Tried running the rebuilds, but item still only shows under Your Items. Only time I can get it into the list under the Showcase tab is by updating cover_image_id (Get it into the list, but still no cover image)

I havent dug into the source (don't know where to begin frankly, newish to XF as a whole (started working here a year ago, did a couple basic things like XF updates, installing new add-ons, etc))

Regards

Peter (posted from Kram's account)

 

[Bob]

AMS (Article Management System) has a built in XenZine importer. If you guys are not currently using Showcase for anything in production, I'd suggest having kram contact me to swap out Showcase for AMS and then you can simply import XenZine into AMS. AMS is better suited for Articles than Showcase is.

As for manually importing content like you are attempting to do, you are going to run into a TON of problems because you are not handling the conversion of data that is associated with content types (attachments are one of those things). There is a TON of import handling that needs to happen programatically that you can't do manually. All the issues you are having with attachments is because you are not importing and converting the attachment data properly (nor are the post important functions like updating cover images and embedded attachments being handled). There are specific ways to handle importing of content.

I'd suggest either swapping to AMS or purchasing an AMS license so that you build an importer for XenZine to Showcase based on the XenZine to AMS importer (you couldn't just copy and paste it, you'd have to do some things a bit differently, but for the most part, you'd see all of the things that have to be accounted for and how things like importing attachments from one system to another system works.

 

[kram242]

Awesome, sounds like an even better plan!
We are using Showcase (for Projects | OpenBuilds ) and was going as far as considering moving "projects" to a new XF install (since we cant run two instances of Showcase) - didnt know about AMS though - that might just be what we were looking for! Will investigate and likely purchase AMS shortly!

 

[kram242]

Right, talked it through with Mark.

Looking at your AMS demo, Articles | XenAddons - XenForo Add-ons & Custom Development its not quite in line with how we used Xenzine, if you could spare a minute, can you checkout our old use case of Xenzine here: https://openbuilds.com/ - people post their machine builds, and the "pinterest" style theme acts as a quick way to build the homepage.

Once you drill down into a Build: OpenBuilds MiniMill for example, the system relies heavily on the tabs on the build, to help users structure their contribution into a building a parts list, etc (Which in Xenzine is "Custom fields"
We also have the Files and Drawings tab (again Custom Field in Xenzine) for users to share files related to their build.
Discussions tab links to a XF forum (created automatically)

If you look at our specific use case, does it come close to what AMS can offer us?

Is there any chance we can get a trial copy of AMS to play with? Just to see how the user perspective and customisability is?

 

[Bob]

You can't really look at a Demo of any of my addons and understand what the addon can do as they are all highly configurable. For the most part, Showcase, AMS, RMS and UBS all share a lot of the same features (custom fields, review fields, reviews, comments, discussion thread integration, prefix system, tons of options, and then all have the same layouts (List View, Article View, Tile View, News View, News View II, Grid View) as well as Modular Layout functionality.

Showcase and AMS are 90% Identical in features and architecture. What exact features from Showcase are you concerned that might not included with AMS?

 

[kram242]

Well, the above answered most of my concerns.
Just one question to confirm: The importer will grab our existing custom fields across too?
(Then tomorrow Mark will push the purchase so I can test in the lab )

 

[Bob]

The importer will grab our existing custom fields across too?

Yes...
Categories are imported into AMS Categories
Articles as imported into AMS Articles
Attachments are converted into AMS Attachments
Custom Fields are converted into Custom Article Fields
Reviews are converted into Article Reviews.

This is a screen shot from several years ago, but nothing has changed since then, so it looks the same.

 

[kram242]

Whoohoo! Tomorrow's gonna be a fun day then!

 

[kram242]

Bought AMS this morning

Spun up a fresh clean install of Xenforo with Xenzine on my localhost, and created a quick setup with two test custom fields (attachment data and table - both types actively in use on our Xenzine implementation - must-have for successful migration)



Then proceeded to create a test article in Xenzine:


(Note, custom fields tab's present on the Xenzine article)

Next, fired up the importer:



Oops, 0 custom fields

Apart from that though, article came across looking mostly intact (images, etc still in the content)




Also, noticed though that the cover image changed:

On Xenzine:
,

but on AMS:

(Grabbed the image I had uploaded into content)


So, guessing I am doing something wrong but custom fields didnt come across, and cover image changed

 

[Bob]

and created a quick setup with two test custom fields (attachment data and table - both types actively in use on our Xenzine implementation - must-have for successful migration)

Neither one of those field types are Core XF supported Fields Types. They are both custom specifically to XenZine internally. Only Core XF supported Field Types are importable and imported (for obvious reasons).

So, guessing I am doing something wrong.

Na. AMS only imports Attachments not the non xenforo supported custom Article Image/ICON (nothing xenforo standard, a one off custom function instead of using core xenforo standard components). Since XenZine wasn't using one of the attachments as the cover image, there is no cover_image_id in the schema that can tell the importer to set a specific attachment as the cover image. The Post import steps will set a cover image for articles that have image attachments, but no cover image set (it fetches the first attachment in the collection). There is a "Set Cover Image" function that Authors (and Moderators) can use to set a specific attachment as the cover image (showcase people loves this as they can continuously change the cover images for their items).

So no, its not that you are doing anything wrong, its just a matter of some NON STANDARD content in XenZine that is not importable.

 

[kram242]

that is not importable.

Ahhh man!

Remember when above, I asked:

Once you drill down into a Build: OpenBuilds MiniMill for example, the system relies heavily on the tabs on the build, to help users structure their contribution into a building a parts list, etc (Which in Xenzine is "Custom fields"
We also have the Files and Drawings tab (again Custom Field in Xenzine) for users to share files related to their build.
Discussions tab links to a XF forum (created automatically)

If you look at our specific use case, does it come close to what AMS can offer us?

(; Was really counting on you actually checking out our implementation... (before continuing the sales pitch) also, why asked for a trial? $120 down (Bought the Early adopters to try and support you) and unusable for our use case (The service we provide our users ALL depend on those two field types)

I'll point Tenants (Xenzine author) at this thread too

We really do need to get off Xenzine though, so, do you have any suggestions to help us out?
Any way to grab the table data from the Xenzine tables and just create a table at the bottom of the content?
Any way to convert Xenzine attachments to attachments that can be used with AMS?

Really stuck in between a rock and a hard place

Please do go and spend 5 minutes on openbuilds.com and familiarise yourself with our usecase / needs so we can avoid this happening further in this discussion

 

[Bob]

I'll point Tenants (Xenzine author) at this thread too

Pretty sure he is no longer developing for XenForo (too busy with his job and family). He was actively looking for another developer(s) to take over his addons. I doubt he is going to be able to help you anyway as the issue you are facing involves 2 custom fields types that are non XenForo standard (they can't be imported into XenForo, XFRM, XFMG as custom fields because they don't exist). You'd first have to extend the addon and implement those fields in the addon being imported into (which is a hell of a lot of customization).

Any way to grab the table data from the Xenzine tables and just create a table at the bottom of the content?

You are asking about a non standard Custom Field Type named "Table" (as in an HTML Table). To import a non standard field, you would have to modify the addon to implement that field yype and then the modify the importer to handle the conversion of that specific field type OR convert it from a custom field to data stored in the article table.

There are 2 approaches that come to mind, both of which are going to require extensive custom development.

Approach 1. Modify AMS by implementing a new Custom Field Type named "Table" that matches the "Table" Field Type in XenZine. Once that is done, you will need to modify the importer to handle the importing of the "table" Field Type content into AMS.

Approach 2. Modify AMS by creating a new blob field in the Article Table to hold the date for the "table" Custom Field Type. Then modify the importer to import the field data into that blob field in the article table. You'll then need to implement how that data is being fetched and displayed. This would be by far the easiest of the 2 approaches to implement.

Any way to convert Xenzine attachments to attachments that can be used with AMS?

The Importer is already importing XenZine article attachments as AMS article attachments. I think what you are wanting to ask about is the non standard Custom Field Type that allows attachment uploading. Similar to the above, there are 2 approaches that come to mind, both of which are going to require extensive custom development.

Approach 1 is the same as above.

Approach 2 would be modifying the importer to convert those Custom Field attachments into Article Attachments. This would be much easier than approach 1.

NOTE: You need to make sure that if you do any of this, you do it properly by Extending AMS via an Addon to AMS, not making customization directly to the addon itself (as that will cause problems when it comes time to upgrade).

Either way, you are facing A LOT of custom development no matter what addon (XenForo Threads, XFRM, XFMG, Showcase, AMS, etc) that you would like to import XenZine content into as those 2 field types are not supported in Core XF, XFRM or XFMG and most addons only have the same Custom Field Types as core xenforo has.

I'll ask around and see if any of the developers I know (and trust) have anything close to Showcase that has both of those non standard field types and a built in XenZine importer that includes importing those 2 non standard field types.

 

[kram242]

Hello Bob,
First I would like to say thank you so much for taking the time to look into this for us and for the helpful responses that we hope will get us up and running for many years to come.

Looking over your suggestions, Approach 2 seems like it would do what we need.

Attachments:
On your test article at your private development site: LINK REMOVED!, there is a good example of a Files Tab - this is very similar to our existing "Files and Drawings" tab in Xenzine.
"Approach 2 would be modifying the importer to convert those Custom Field attachments into Article Attachments. This would be much easier than approach 1. " - I assume this one would then bring existing Xenzine attachments in, such that we can display them on the Files tab, like in this example? If yes, then thats what we want!

Parts List Table:
"Approach 2. Modify AMS by creating a new blob field in the Article Table to hold the date for the "table" Custom Field Type. Then modify the importer to import the field data into that blob field in the article table. You'll then need to implement how that data is being fetched and displayed. This would be by far the easiest of the 2 approaches to implement. "
I have not dug in very deep into this: I note that Xenzine uses Blobs to store the Table. Opening up the blob, reveals a relatively easy structure:

a:2:{i:0;a:4:{i:0;s:2:"10";i:1;s:9:"PartName1";i:2;s:12:"PartLink1com";i:3;s:8:"Comment1";}i:1;a:4:{i:0;s:2:"20";i:1;s:9:"PartName2";i:2;s:12:"PartLink2net";i:3;s:8:"Comment1";}}

Possible we can just convert that to BBCODE Tables: BB Code Manager: BB Code XML Repository and place the BBCODE right after the Article content (Inserted into the actual content = no need for any extra fields / blobs, just let the importer convert it to BBCODE strings:

Qty	Part Name	Part Link	Comment
10	PartName1	PartLink1com	Comment1
PartName2/TD][TD]PartLink2net	Comment2

20[/TD]

We would love to ask you for you help in making this happen and at the moment we are kinda stuck without it.
If you could let us know, that would be great. We look forward to working together into the future implementing all the cool new mods you are working up.
Thanks again Bob for all of your help and for making this software available to us all.
Mark

 

[Bob]

I assume this one would then bring existing Xenzine attachments in, such that we can display them on the Files tab, like in this example? If yes, then thats what we want!

Yes. If the file type is a FILE, it would display those files however you condfigured the addon to display them (tab, below item, sidebar block etc). If they are image, they will display with all the other image attachments.

Basically, there would be TWO functions in the importer that import attachments. One (the existing one) stays the same. The new one that you develop, would convert custom fields attachments into AMS (or Showcase) attachments (which the end result would be a merged set of attachments). You might even be able to COMBINE the 2 functions (altho, I'd do them separately if it was me).

As for the TABLE BBCode, I don't really have a good answer for you. If I were in this situation, I'd probably TRY to do one of the 2 approaches that I mentioned. I've never done it, so its just theory. You'd have to do the R&D and see which approach might work for you. It could end up being their neither approach is viable, which means you'd need to come up with another approach.

We would love to ask you for you help in making this happen and at the moment we are kinda stuck without it.

Unfortunately, I am not in a position to take on any custom development right now. I have a very demanding full time job a Family and a ton of XF2 Development going on right now. Once XF2 development (the initial rewrite phase) is over, then I will be in position to start taking on clients again. I can ask around to see if any of the developers I know/trust might be available to help out.

 

[kram242]

Hi Bob, thank you for the response and the offer to ask around for help. At the moment its really not an emergency (its working fine knock, knock, ) We completely understand the super busy aspect. How did we let ourselves get this way it seems every year it gets even worse, there never seems to be time for anything else but work for us. I guess it can be looked at as a good thing but it does overwhelm from time to time.
Well again thank you for your help and we will keep working at this until such time as your schedule clears up a bit able to jump in and lend a hand. We are hopeful that if we get stuck we can shout out to you from time to time as we work at this?
Have a great day my friend
Mark

 

[Bob]

We are hopeful that if we get stuck we can shout out to you from time to time as we work at this?

Of course