EG Redesign - Ideas and Suggestions?

[ProxCenBound]

While I was writing the above comment I asked on Discord, and it was Tasp who did that. They said "Probably pretty easy to get at least ballpark numbers for each article. I'll play with it when I get back to my personal machine". So we might be able to find out soon.

Tasp got back to me on Discord yesterday. Their findings are below. Many thanks to them.

Note that topic pages are here included as articles, but stubs that appear only at the bottom of other pages are not included.

-----

Preliminary results



Size,Count
100,1104
200,805
300,480
400,328
500,246
600,184
700,154
800,133
900,120
1000,86
1100,78
1200,53
1300,66
1400,37
1500,55
1600,44
1700,52
1800,37
1900,36
2000,36
2100,23
2200,27
2300,15
2400,25
2500,12
2600,13
2700,14
2800,13
2900,11
3000,8
3100,4
3200,4
3300,8
3400,8
3500,7
3600,5
3700,7
3800,4
3900,4
4000,5
4100,3
4200,5
4300,8
4400,1
4500,4
4600,1
4700,1
4800,5
4900,3
5000,4
5100,2
5200,6
5300,3
5400,1
5500,3
5600,2
5700,5
5900,1
6000,3
6100,1
6200,2
6400,2
6500,2
6700,3
6800,1
6900,1
7100,2
7400,1
7500,2
7700,2
7800,2
8000,1
8200,1
8300,1
8400,2
8500,1
8700,1
8900,2
9000,1
9500,1
10500,1
10900,1
11600,1
11900,1
12200,1
13000,1
13600,1
14000,1

Top 10 longest articles:
"Technocalypse", 13959
"Red Star 'M'pire", 13572
"Solsys", 12965
"Glossary", 12179
"Inner Sphere", 11810
"Version War", 11521
"Corona (Iota Piscium d)", 10818
"Beneficence ", 10453
"Bubblehabs", 9487
"Pacifica", 8993

The shortest article is "310213321001"

[Drashner1]

I would like to ask, does the OA CMS have any sort of backup system? Or is our only way of restoring lost/damaged articles to use web archiving services?

The OA site and CMS are both run on Google's server service and are backed up automatically. If something were to happen it would require Trond's assistance to restore things, but the information itself would not just be lost if something like a power outage happened or something.

Hope this helps,

Todd

[Avengium]

How can I move this topic forward?

Is there anything we have discussed to do regarding this redesign?

[1of3]

For people who want more customization, there are lots of skins: https://www.mediawiki.org/wiki/Category:All_skins

I support the idea of a wiki. I think mediawiki is overkill for pretty much any use case.

Drashner suggested that such a migration is only feasible if there is automation, to which I very much aggree. But that's pretty hard to do with media wiki as it uses a rather complicated database structure.

Rather, I would suggest Gollum.

https://github.com/gollum/gollum

Which is a web interface over a git repository. It also works with several markup languages, though markdown (supported in several dialects) is most common. Now, I do not know what the current format of EG is, but assuming it's html or we can get html, automated tools for conversion to already markdown exist.

What would be required is a skript for setting intra wiki links.

When that is run, since the underlying structure is a git, we could simply check in pages to the wiki.

I'm willing to try this, if I can some pages as they are currently stored in the CMS, best some pages that currently link to one another and have category / subcategory / article relationship.

There are various other wiki softwares as well.

[Drashner1]

How can I move this topic forward?

Is there anything we have discussed to do regarding this redesign?

I think before we get much further with this that we should run it by Trond. As our Webmaster and resident website guru he has the expertise to consider how this might (or might not) work with the existing website (the EG is the core of both the OAUP and the website - but is not the only part of the website).

I am just back from vacation and still digging out from a backlog of both OA and RL items. I have some things I need to discuss with Trond later this month in any case and can raise the idea of an EG wiki with him as well.

Or - Others taking part in this discussion can reach out to him directly. We have other members of both the 'staff' and the Managing Board taking part and it's not like I have a monopoly on talking to him. The last time I spoke to him he was very busy in RL, and I don't know that that will have changed. But if he doesn't have time/bandwidth to respond then I imagine he either won't or will make that known.

Or I can touch base with him later this month and ask. If the idea has landmines or other issues that prevent it, I'm sure he will let us know. Conversely, if it is workable/positive, he can let us know that and possibly suggest things we should look for in a wiki package or the like.

My 2c worth,

Todd

[administrator]

I would like to ask, does the OA CMS have any sort of backup system? Or is our only way of restoring lost/damaged articles to use web archiving services?

We have automated DB backups, and I keep offline backups in a few other locations which I take every few months. This accounts for DB loss events.

The CMS also uses a journaling database model, meaning that we keep a log of all previous versions of every page, and can revert pages if that ever becomes an issue. In 15 years, it hasn't really come up.

[administrator]

How can I move this topic forward?

Is there anything we have discussed to do regarding this redesign?

I think before we get much further with this that we should run it by Trond. As our Webmaster and resident website guru he has the expertise to consider how this might (or might not) work with the existing website (the EG is the core of both the OAUP and the website - but is not the only part of the website).

I am just back from vacation and still digging out from a backlog of both OA and RL items. I have some things I need to discuss with Trond later this month in any case and can raise the idea of an EG wiki with him as well.

Or - Others taking part in this discussion can reach out to him directly. We have other members of both the 'staff' and the Managing Board taking part and it's not like I have a monopoly on talking to him. The last time I spoke to him he was very busy in RL, and I don't know that that will have changed. But if he doesn't have time/bandwidth to respond then I imagine he either won't or will make that known.

Or I can touch base with him later this month and ask. If the idea has landmines or other issues that prevent it, I'm sure he will let us know. Conversely, if it is workable/positive, he can let us know that and possibly suggest things we should look for in a wiki package or the like.

I'm not amazingly keen on converting to a wiki like Mediawiki, at least partly because it would be a lot of work for no obvious gain. Alternative models like Gollum and various other things that are out there are more interesting, but I don't currently see the specific advantages. I am open to having a conversation that includes this and really any other ideas, so let's get into the weeds.

The main issue in doing an actuall conversion would be converting from our wikitext format (which is Mediawiki-like, but not exactly the same) to whatever the new system uses. We do have some augmentations in there that have allowed editors to customize the way pages look in ways that weren't possible in mediawiki at the time (and are still bit janky today). These would need manual conversion and possibly a bunch (probably a lot) of a CSS fighting to make it work. Moving the bulk of content over would just be a matter of throwing together some scripts around the existing regex libraries I have to make updates to the system, but going through and making everything look right would be a very substantial amount of effort.

Some issues with using Mediawiki:

• Outside of substantial customization, all structure in most wikis is manual, meaning that there are no semantics associated with individual pages articles that can be queried and used to do various things. So, we can't easily extract a list of articles authored by a particular person, we can't extract copyright notes in an automated fashion, and we can't populate the timeline automatically as a view over the wiki. You get date, title, and version metadata, but not much more. There are some janky workarounds and you can implement these things with custom code on top of a Mediawiki site, but doing so can be a lot of work, and tends to be pretty fragile over time. We also have features on the EG wishlist that take advantage of our more structured DB, though granted, they've been on the back burner for some time.
  
• The options in Mediawiki for look and feel aren't great. Different wikis have different pros and cons, but the system we have is more flexible than any wiki I'm aware of, in large part because we control the templates and stylesheets directly.
  
• In general, any customizations will be fragile. That is, as they post updates to the framework (which we would need to stay up with for security reasons), things will need to be updated and maintained, and I know from experience this can be quite time consuming.
  
• As it is the EG ties into the main site in a bunch of ways that would be lost if we shifted to a generic wiki solution. 
  

Alternative document schemes like Gollum, or something like a giant Obsidian repo are interesting, but again, I don't see an obvious big advantage, though I'm open to too suggestions. If the OAUP moved to a more open editing process that lines up more with how wikis typically operate, then those models could well make a bit more sense, but as things stand, it's not clear what this would gain us, particularly given the effort required.

Ignoring the specifics of whether a wiki is a good idea, though, it's probably best to think about this from the perspective of "what are the pain points", and "what are the limitations" as far as the user experience goes. Whenever a technology discussion is lead by "we should use technology X" instead of "we should solve problem Y", you're much more likely to make bad decisions.

Given that, then, what does the current system suck at? What are its weaknesses? Given a concrete list of those items, it would make it much easier for us to weigh up changes to the existing system or shifting to a new model such as a wiki or other document repository. Todd and I already have quite a long wishlist, and I saw mention of concerns around the version history of articles; this feature exists (we keep a record of every change over time), but the UI isn't present - I built this functionality in back at the beginning, but got feedback that it wasn't really important, so I never finished the UI. I also saw mention of some look and feel requests; though these are generally quite straight forward. What else do folks want to see changed?

As far as contacting me goes, you can find me on Discord fairly easily as trond.nilsen. I'm online there most of the time when I'm not sleeping. I don't usually come into the forum unless something in Discord prompts me to do so or Todd nudges me about something (I like forums in principle, but I never manage to keep up with them and have largely given up trying). The reason I'm looking in on the thread tonight is because someone bugged me there.



(Edit - I just realized I posted this as administrator; I can't change that in an edit, but will switch back to my account going forward. Auto-login got me).

[administrator]

So - as one of the admins who has been very resistant to the idea of converting to a wiki - and to throw my 2c into the mix on this:

Taking the easiest bit first - IIRC someone on the discord or here ran a script or the like to do a page count and it came back that we have something around 4500 pages in the EG.

I wonder how hard it would be for it to measure article length in word count by that same means.

While I was writing the above comment I asked on Discord, and it was Tasp who did that. They said "Probably pretty easy to get at least ballpark numbers for each article. I'll play with it when I get back to my personal machine". So we might be able to find out soon.

I ran some DB queries - we have 5213 articles and 340 topics, totalling around 3.5 million words. The word count is a bit vague because it's non-trivial to separate actual words of content from some of the markup and stuff like image names.

[administrator]

Another possibility that might reduce the overall number of articles to move is to reimagine how information is organized in the EG.

As a major example - the History Topic contains some 33 sub-Topics - out of around a total of 300 - so something like 10% of our Topics are devoted to just that one aspect of the setting. Replacing all that with a single very large page, perhaps with 'table of contents' links to jump down to specific area, search functions, and possibly other things would hopefully be a better end-user experience (of course all the actual entries would still have to go in....).

Something I would love love love - but don't know that a wiki can do it - is to make a timeline that we can take chunks of and insert into other pages - creating 'sub-timelines' in other articles - that are still linked back to the master timeline. So a change to the master is automatically reflected in all sub-timelines. I'm told this is possible and not all that hard, but it would need Trond to program it up. If a wiki could just do something like that baked in, I would be very impressed.

Todd

You could do this with our existing timeline but you'd need to define a rule for which content you want to include in a given sub-timeline (eg, this timeline shows articles that descend from this topic, or articles in this time window, or whatever), then we'd need to add markup to let you specify that in the editor.

[1of3]

How does the timeline currently work?