Any way to avoid the mod-rewrite side effects...

[marksman]

I set up my directory with the urls to be rewritten. The problem with this is any invalid links are just redirected to the main page. Sometimes even accurate links seem to get looped inside themselves returning to the main page and creating nonsense urls, which potentially impact users and definately confuse the search engines.

Short of just scrapping the re-writing is there any other way to deal with this issue?

If I have a directory structure of a1->a2->a3 and someone goes to b1 then it goes to b1-a1-a2 etc... it ends up quite the mess.

Let me elaborate a bit too.. I consolidated a bunch of seperate directories into a single one.. so I am using 301 redirects from the old subdomains to the new url and this is where the bulk of the problems are..

I don't really want to give up the rewritten urls so I am tempted to just ride it out, but would love any alternatives...

[kickass]

Though you probably are already aware of the answer, given what you've written, the mod rewriting works on clean installs just fine, it's one thing that seldom appears as a problem on this forum. It's vaguely possible one of the tech heads could help you straighten it out given some boredom and a lotta spare time, but I really think the best thing is to start over.

If all the directories have different databases and you've tried to combine the directories into one without combining the databases, that's probably what's confusing things, and is what your elaboration seems to suggest. If you want everything in one directory, you need to somehow import the other databases into the main one.

My suggestion is to back up all the databases and use the backups along with phpmyadmin, to create a new merged database for a new install and see if maybe that gets you what you want. And if I'm totally off base I'm sure someone will correct me.

[Ap0s7le]

BJ is correct.

It doesn't sound like a rewrite "problem" but more of a lack of planning.

Make sure all your redirects are correct as well.

Good luck.

-Casey

[offmaster]

I got the same problem with a basic/clean install. Actually it becomes a problem if someone makes a wrong link to my directory (deliberately or not)

exemple :

{url no longer in service} is a true cat it works well, no problem

but

{url no longer in service} (this one doesn't exist) but it will display the sub1 content and all the links of the subcats as well. Spiders will crawl it and will make dup content.

Is there a solution to limit the rewrite to the real cats only ?

Thanks for your help (and sorry for my english)

[seothatworks]

The quickest way is to make a permanent redirect in your .htaccess file from the wrong url to the correct url. Or, ban search engines spidering the wrong url via the robots.txt file.

[offmaster]

I just can't anticipate all the wrong url :shock:

[seothatworks]

You can't, but I don't think there will be more than that one, right? I have so far no problem with all my directories. You can quickly find out in your site's log and take corrective measures.

Of course I know that preventive measures would be much better, the point is how hard will they be to code if possible, and if it is such a big problem worth of fixing (maybe it is, I can imagine some competitor wishing to damage your site might attempt it, on the other side it would have to be a rather rare case with unpredictable results - not sure how much harm that can do).

[marksman]

Content visible to registered users only.

This is what I have been doing for the most part.. Although I have encountered a bug where sometimes even when I go directly to an existing url it stacks me back on the main page.

Haven't been able to entirely debug this one...

Biggest problem right now seems to be yahoo.... It is the only one still having a problem as far as I can tell...

I agree the issue was not a clean install and using redirects.. On top of that I did not redirect every single url from the old site and instead did it by paths... This is where the problem comes in...

I do still think there is a potential issue here for sabotage from people just making up invalid links from external sites and letting spiders crawl there.. This could potentially turn your front page into its own duplicate content and cause problems. There are concerns with places like google penalizing all duplicate sites at present.

Regardless, I seem to have gotten a lot of my pages now showing up in all 3 serps so I will just keep track of yahoo and its craziness. Not sure why Yahoo is so happy to stack this urls... Wish I knew where/how it was even getting them...

As I said though I will just block them out as I see them...

[kickass]

I can't say if this is your problem but I thought it was worth pointing you in this direction . . .
{url no longer in service}

[serverunion]

I think a 301 redirect in the PHP code for invalid directories would be a nice addition.

[marksman]

Thanks KA... I actually saw that before I set up the directory, so when setting it up I disabled the counter to avoid the bot problem.

It seems to be fairly isolated to only yahoo now.. so it is not really a big issue for me any longer...