Not Planned "Soft" moderation on failed URL check (& solution if not implemented)

[frm]

If a URL check fails, the item will go into moderation. This will take it out of public view, rendering a 403 (Forbidden). This could potentially impact the ranking the page once had if it's crawled, and have to re-earn its position/reindex again when it's back up, if not caught in time.

It would be nice if on a "failed URL" check that it goes into the moderation queue, but is a "light" moderation where it's still publicly visible and accessible by search engines.

This way, you can manually vet failed URLs as they are, by my best guess, typically non-responsive for a duration of time that signifies they're "down" (or a random Error 500). However, every manual check (10s so far, with about 2-3 done daily), shows me that the site is still live. (If it were dead, I'd put it in a "soft delete" state and reconfirm it a week later for further action)

Solution if not implemented / you don't want 403s on link directory listings while in moderation queue​

To solve this for now, I added this permission to both the Unregistered / Unconfirmed (Guest) and Registered groups.



Then, I created a new phrase: xa_ld_custom_awaiting_url_check with the text Awaiting manual URL health check

And searched for the templates in LD containing awaiting_approval_before_being_displayed_publicly and awaiting_approval

To find the templates:

• xa_ld_item_list_macros
• xa_ld_item_wrapper_macros

To replace awaiting_approval_before_being_displayed_publicly and awaiting_approval with xa_ld_custom_awaiting_url_check.

However, I didn't want it to visually appear moderated.

So in xa_ld_item_wrapper_macros
I changed:

<dd class="blockStatus-message blockStatus-message--moderated">

to

<dd class="blockStatus-message blockStatus-message--urlcheck">

To get the block status icon working (not the shield), edit core_blockstatus.less (in your child style) and added this (1st line is already there to give a guide on placement):

    &--ignored:before { .m-faContent("@{fa-var-microphone-slash}\20"); }
    // CUSTOM LD MODERATED URL CHECK
    &--urlcheck:before { .m-faContent("@{fa-var-tools}\20"); }

Then, xa_ld_item_list_macros needs edited.

Find:

                        <xf:if is="$item.item_state == 'moderated'">

And change to the custom phrase (if you haven't already) and edit the CSS styling:

                            <li>
                                <i class="structItem-status structItem-status--urlcheck" aria-hidden="true" title="{{ phrase('xa_ld_custom_awaiting_url_check')|for_attr }}"></i>
                                <span class="u-srOnly">{{ phrase('xa_ld_custom_awaiting_url_check') }}</span>
                            </li>

You'll replace status--moderated with status--urlcheck (along with the custom phrasing).

But, we need to edit 1 more CSS file to get rid of the moderated shield icon and italicized title.

Edit structured_list.less

Find:

    &.is-moderated
    {
        background: @xf-contentHighlightBg;
    }

And you might need to edit the background color of it. On my style, I did not.

Find: .structItem-status and insert the urlcheck CSS in the same fashion to change the icon.

    &--downvoted::before { .m-faContent(@fa-var-thumbs-down, 1em); }
    // CUSTOM LD MODERATED URL CHECK
    &--urlcheck::before { .m-faContent(@fa-var-tools, 1em); color: @xf-textColorAttention; }

I am using grid view, so the last modification I made was to xa_ld_grid_view_layout.less and I commented out the italic style that the title would have otherwise had in moderation.

        &.is-moderated
        {
            background: @xf-contentHighlightBg;
            
            .grid-caption .grid-item-heading
            {
                //font-style: italic;
            }
        }

But if you're using tile view, you'll need to comment it out in xa_ld_tile_view_layout.less:

        &.is-moderated
        {
            .tile-caption
            {
                background: @xf-contentHighlightBg;
                border: 1px solid @xf-textColorMuted;

                .tile-item-heading,
                .tile-item-heading a
                {
                    color: @xf-linkColor;
                    //font-style: italic;
                    text-shadow: none;        
                }

Now in grid view, the item will still appear, but show that it's awaiting a manual URL check.



These of course will remain in your moderation queue to check the URL.



It's up to you to approve them, which would make them regular listings again, delete them, or convert them to threads.

However, your pages won't 403 anymore if they're ever randomly requested/crawled while you're away; you will have control over that when you get back to moderating your forum.

 

[frm]

I honestly don't think this is possible looking at

                        <xf:if is="$item.item_state == 'moderated'">

Unless XenForo is also extended to have an item_state of "urlcheck" to enter the moderation queue. But then, you would still need to view moderated permissions, I would think.

This might be a headache to implement the way XF is. I don't know.

But, the solution works for me and I'm happy to answer any questions if you're having trouble editing your LD listings so they don't disappear overnight.

Another solution would to be disable the URL check cron job and only run it when you want to, then quickly go through the moderation queue.

 

[frm]

The first post was wrong in that it'll return a 403 (Forbidden), not a 404.

It's unclear how Google handles 403s. There's conflicting information on whether they'll revisit, and how many times, before they're eventually deindexed. Reports back to 2012 show that it could be one time, but webmasters now are also stating they'll hit it until it's available again. I would still err on the side of caution and state that if they hit it, they could hit it again an hour later, and decide to deindex it as we do not know how Googlebot works with crawling and indexing for certain. It's better to be safe than to be sorry.

 

[Bob]

I've marked this as "Not planned" as I have no plans to make any changes to how the failed URL Check works.

Letting you know now so that you can pursue custom development (or to do customizations on your own).