DMOZ Import Mod

[f1gm3nt]

This version should get rid of the column errors. If it doesn't work right please post what other mods you have, if any, or any other kind of information.

DO NOT DOWNLOD THE FILE BELOW unless you want a useless outdated copy that does not work.. Please download the one above because it has fixed the column errors.

Can an admin please remove the file below?

[Ap0s7le]

Nah, don't post flames. No point, and Josh is doing everyone a favor so only Thank You's are accepted.

Thanks Josh, and I locked the other thread so people will start coming here instead.

-Casey

[f1gm3nt]

Thanks, this next one should be easier for everyone to use. I'm setting it up to where you want need to use phpMyAdmin....much... :roll:

[Ap0s7le]

Josh, if you need help yell.

Or just get a IM!! haha

The front end could be web based. Enter the Structure path you want, DB User/Pass and DB name.

It would go ahead, start the whole process. Log it's progress on screen as Cats / Subs / Links but log details in a TXT file.

Give the option to Append a PHPLD install, or a Fresh install. So it would know if the cats it's about to add are the only ones in there. If you opt for the append, it could then check the DB for existing fields for duplicate data.

Or something like that... I'm rambling... Going to lay down and watch a lil tv...

later

-Casey

[f1gm3nt]

Well, it has always checked to see if the categories and links are in the phpLD database so it knows which to add. If it's already there it doesn't add them :)

Right now I've got it set up to where you create the tables and edit the two files, class.dmoz.php ad class.sql.php.

class.dmoz.php you just change to the directory that you want.

You don't have to add the first link in the DB. The script does it for you now.

What kind of details would be placed in the txt file? It seems like a good idea but not sure what kind of info could be placed in there.

In the future I want it to mooch off the phpLD install. That way it gets the info it needs.

You should know I don't use IM, I'm just a crazy linux guy who doesn't back up his data :P

If anyone has any suggestions or comments let me know.

-Joshua

[f1gm3nt]

Just want to give everyone an update. Right now I've got a bug that I'm working on. Once this bug is fixed it should work and I'll release a new version of it. The bug is when it goes to check on categories and doesn't do it right and it ends up adding the same category a bunch of times.

-Joshua

UPDATE This bug has been fixed. The problem was that it would check the title and not the URL title. This script will replace certain character that it finds in the URLs and replaces it with the correct letter or whatever you want it to be.

I want to test this a little more before the final release of it. If you want a test copy let me know by posting the request for a TEST copy or PM me. It should work but I am testing it to make sure there aren't any more little bugs lurking around to piss me off later :evil:

-Joshua

[jminscoe]

I would like a test copy

[jminscoe]

:D So far it has crawled and I have about 40000 links so far I haven't converted them yet but this copy seems to be going without a hitch so far will keep you posted :)

[f1gm3nt]

:shock: that's a lot of links!

I haven't tested it with that many yet, but keep me posted. Once it's finished crawling converting might take some time. I haven't tested it a lot with large links like you're doing, but if the convert.php page times out you should be able to reload without any data loss or any duplicates going in.

Keep me posted either by here or PM me.

-Joshua

[jminscoe]

:D :D Ok it converted most of them
the following is what it did

Table Records
PLD_CATEGORY 6,817
PLD_LINK 62,176
category 6,910
dmoz 6,911
links 66,120
One more question can I delete category, dmoz, links now or what

[f1gm3nt]

Yea, after you done with everything I would go ahead and delete all the files you uploaded and also the three tables(saves some space).

I am kinda wondering why the row counts are different for the links. I'm assuming that it doesn't import them all :cry: I'll have to look into that.

About how long did all that take?

-Joshua

[jminscoe]

On my pc it took all night to import :( to convert about 30 minutes :D I just had to refresh a few times I think the difference was because it couldn't connect b/c when I checked it said page not found

[f1gm3nt]

heh, that's awhile! Let me knowif there's anything I could improve on it. In the future I would just like for people to just upload and run it, but for now it doesn't.

-Joshua

[jminscoe]

I am not that savvy on code but considering I was dmozing on Health I think it did a great job and considering I had over 60000 links to crawl and import what can one expect :)

[f1gm3nt]

perfection? I guess it works, if anyone else wants this thing then PM me and I send it to you.

-Joshua

[Spiderman32]

Minor fix in crawl.php

echo "

Crawl complete! Now run <a href=\"covert.php ---> convert.php

[f1gm3nt]

Yea, I noticed that, it should be convert.php with an N. Long night of working on it. Sorry about that guys and gals.

-Joshua

[oneant]

hi,

first, thank you for sharing your mod to the community.

i tested the mod and it worked great, i then edited the URL of the directory to crawl on the crawl.php and uploaded it on the site and refreshed crawl.php but nothing gets crawled.

what i want to do is append the old database with the new data, is this possible? if yes, how?

thank you

[f1gm3nt]

If you are wanted to append your current phpLD then just upload everything and it should just append your directory. Be sure to back it up first just in case something goes wrong.

If you've read the ReadMe.txt file and edited the two files(class.dmoz.php and class.sql.php) and uploaded them then you should be fine. You of course also have to install the tables that it uses by using phpMyAdmin or some other means.

Once you have finished 'crawling' it will tell you and then you need to run convert.php. This converts all the data and puts it into the phpLD tables. It will check to see if the category is already there and if it is it will not add the category.

Please let me know if this helps or please submit some more detailed information about your problem.

-Joshua

[Jim_Westergren]

I have made this another way with a PHP script I found on hotscript.

Actaully quite simple.

[f1gm3nt]

I've used that before. It doesn't do the same thing.

-Joshua

[Ap0s7le]

Yeah, that script isn't the same at all.

The data becomes part of your directory, meaning when someone submits a site you don't have to wait for DMOZ to accept it, you can do it yourself which is far easier and more customizable in the end.

later

-Casey

[OWG]

Dumb question but is this now complete, What I mean is in the download, have the corrections been made, or will I have to make them?

[f1gm3nt]

Everything works except the link to convert.php points to covert.php

Everything else is fine check out {non resolving url removed} to see the section it crawled.

The readme file will tell you how to set it up, if you have more questions post them here or contact me directly.

-Joshua Estes

[Jeru545]

Excellent script and thanks for sharing it with the community.

I am having an issue and I'm not sure if it is host related (I'm with Dreamhost) or otherwise:

Content visible to registered users only.

I am using the latest version of the script and tripled checked the DBs created and the MySQL settings and all seem to be correct because the first line is being inserted into the DB but then nothing along with the above error.

Any input would be appreciated.

Thanks

[Jeru545]

Actually after some Googling this is a Dreamhost issue.

I will post a work around if I can find one.

EDIT: Well I got as far as replacing the following in crawl.php

Content visible to registered users only.

with

Content visible to registered users only.

and then tried this

Content visible to registered users only.

Both times yielding the following message

Content visible to registered users only.

I think my syntax for the following line in the above code is wrong but have tried many variations to now avail

Content visible to registered users only.

I think the only solution in the case of Dreamhost is to run the script locally and then import into the DB.

[webinv]

I have a couple questions:

1. I was able to crawl and import the categories I wanted just fine (except for having to refresh the page a few times).

2. Now, whenever I delete a category I get an error message that the smarty template could not validate the category (or link) that I tried to delete. It still deletes it, but gives an error.

3. Pagerank will not display for any of the DMOZ imported links (because of the smarty template validation thing?). It will only show the N/A image for pagerank.

Any ideas?

Thanks,

-Jon

[f1gm3nt]

1) You imorted a lot of stuff so it may have just timed out

2) Not sure about this one. It may be an error with phpLD if you are getting errors. I tried to add a category and delete it, but some strange stuff happened. It seems like it just renamed a category.

3) PR I have no idea. I don't really use it so I don't know anything about it.

Sorry I can't be more help.

-Joshua

[agolkar]

Hi :)
thank you for your job and your effort for making this mod

I tried using it on my brand new link directory, but I had no result.
I set up all the stuff as written on the readme file,
when I launch crawl.php everything looks fine (but it lasts about 30 seconds or something like this). Then I run convert.php, says everything is OK. but then no new link was added on my directory :(

I saw on the database the "link" tables has got some links, so it looks like they have just not been converted. On the other hand, the "category" table is empty.

As test, I did not change the directory and I left all as default, I have just put my sql info and that's all.

What's the problem? Thanks

[f1gm3nt]

crawl.php reloads itself. It will let you know when it is done. Try going back to that and just letting it run.

-Joshua

[NikkiBLue]

I encountered the same problem. I followed the steps and let crawl run, then went to convert.php. Arew we supposed to create the directory names first?

[oneant]

i am using the latest versin of phplinkdirectory the crawl was finished and i tried to convert then i got this error message.

1 - Inserted
SQL ERROR:1136: Column count doesn't match value count at row 1


is this compatible with the latest version of phplinkdirectory?

thank you

[f1gm3nt]

I haven't tried it with 2.0 the last version I tried was RC5. When I get some time I'll try to look into it and test it a little. I have a few minor bug fixes so once I try it out I'll let you guys know.

Joshua

[stuw]

I have this problem too, and it's the latest version. :(

Content visible to registered users only.

[f1gm3nt]

It looks like it dose not work with the new version(2.0) Im going to upgrade my test directory and work on this thing. You should be able to snag a new version in the next few days.

In the mean time if there is any thing you want in the next release let me know.

Joshua

[NikkiBLue]

[quote="f1gm3nt"]It looks like it dose not work with the new version(2.0) Im going to upgrade my test directory and work on this thing. You should be able to snag a new version in the next few days.
/quote]

Thanks for taking time to update this script! I can't wait.

[f1gm3nt]

It should work now under phpLD2.0 there is also a minor bug fix that webinv pointed out and that I had been having some other problems with. If you get any problems with it let me know. Also feel free to give any feedback that you may have.

I also changed it so that it just grabs your MySQL info so it's a little easier to install.

[dowload link removed and put in first post]

[NewGenius]

Hi,

>>>3. >>> Just set the Status of the Links to "inactive" and then again to "active". I solved Pagerank-Display-Problem with this!

Wolfgang


Content visible to registered users only.

[f1gm3nt]

Alright, I've been going over some code and I've found where that bug is. I'll have to call some pagerank functions and then insert that data into the database. I'm going to start work on adding some more stuff. The next version I would like for people to just upload it and not have to do much of any thing else.

I'll try to keep everyone updated but this thing requires a lot of my time which is very important to me. I am working almost 24/7 on a business venture that is very demanding of my time. I stop by this forum every few days and I can't really tell you how long it will be before a new version comes out.

Also has anyone used the other DMOZ mod that requires the RDF downloads? If so can you give your input on it? I'm sure he would like that as much as me :D

-Joshua

[FB]

Content visible to registered users only.

Am I missing something on your site? It doesn't look like that many categoies or links?

Just trying to figure out this mod.

[jminscoe]

Yes there are that many on that site its categories and subcategories which have links

[bold]

I get this error when I try to run crawl.php

Content visible to registered users only.

Anyone know what could be wrong?

[FB]

I tried it tonite thinking I would let it crawl all night and got the same error message...I've got my host working on it right now. Hopefully he will have an answer soon and I'll post if I find out.

Kevin

[FB]

for me it was this line:
Content visible to registered users only.

I had to change it to MySQL host name which was Content visible to registered users only.

Now I've run into a different issue...I get this...

Content visible to registered users only.

I am guessing that since I put quotation marks around the table names when I created it, that's the reason it doesn't exist?

[FB]

Ok...got it working...well sort of....pointed the browser to crawl.php and it took about 3 seconds and said done at the bottom of my browser and left me with a blank screen...now I'm guessing that it's not quite that fast and it didn't work quite right:wink:

Any suggestions?

[FB]

Just ran it again to check and it worked...well sort of...crawled it in two seconds, and I got a message to convert.

Content visible to registered users only.

[f1gm3nt]

Are you guys using the new version? I have had that error message before but with a different application I was making. Most webservers only allow MySQL connections localy meaning that remote connections wouldn't work.

If you are having to change that line of the code you may not be set up correctly. It should pull the information out of the config file to use. You might have to give the script the entire path to that file. Check class.sql.php far that.

Also be sure that you have created those tables on that server. This script doesn't do it for you.

Let me know if you can't get it working.

Joshua Estes

[FB]

Yes, I installed the one for phpLD 2.0
I used the same host name to setup the phpLD database...not "localhost".
Just uploaded the program last week sometime. Yesterday was the first time I tried it.
All the tables are created in the database...just like in the instructions..



Three tables with the settings set as instructed.

dmoz
categories
links

[FB]

I even tried changing categories to a smaller size (about 8 links).

The same thing happened..crawled in about 2 seconds, and asked me to convert, but got no links.

[kodakball]

After looking at the php code I noticed an anomaly/error in the convert.php. The error is caused in this line 87 , since i can't post the whole line since it causes a forum redirection bug/error


$parent_id."','2',NOW(),'0','0','0')";

After comparing the data base structure, 2.0 doesn't use
SYMBOLIC int(11) NOT NULL default '0',
SYMBOLIC_ID int(11) NOT NULL default '0',
anymore , so a quick fix that worked for me was remove the last 2 '0' from the code above, and viola it working now, a quick but dirty fix.

I hope this works for other people.

[FB]

Content visible to registered users only.

If you post it with [*code*] [*/code*] tags around it, it won't be a problem to the forum. remove the "*" :wink:

[FB]

I take it that this is just for the conversion end of the program? I simply cannot get it to crawl anything...it just crawls a category in about 2 seconds
with no results.

[kodakball]

Content visible to registered users only.

Well just post the first 20 lines in class.dmoz.php, and class.sql.php and i can compare it to my working version, and tell if something is not right.

[FB]

Content visible to registered users only.

Well just post the first 20 lines in class.dmoz.php, and class.sql.php and i can compare it to my working version, and tell if something is not right.[/quote]

for whatever reason, I cannot submit it between code tags??

[f1gm3nt]

Content visible to registered users only.

crawl.php will 'crawl' the dmoz site and convert.php will convert all the data to the phpLD format. The reason there are two files like this is because I just to use it to import links into a different links script.

[f1gm3nt]

Content visible to registered users only.

I haven't tried it on anything except the localhost. The next version I'll try to leech the SQL functions that phpLD has so it should work if phpLD works.

-Joshua Estes

[f1gm3nt]

Content visible to registered users only.

I just checked out the tables for 2.0 and I had to add some more stuff. If that works for you then great. I'll look into the code some more to see if I've made any errors.

-Joshua

[f1gm3nt]

Content visible to registered users only.

Well just post the first 20 lines in class.dmoz.php, and class.sql.php and i can compare it to my working version, and tell if something is not right.[/quote]

Those are class files and if you make changes to those let me know. I may include them in the future. I'd check your config. Like I said above I haven't tested it on anything but localhost. If it's different then localhost then I'll have to check out more of the phpLD functions.

-Joshua Estes

[f1gm3nt]

Sorry about all those posts guys. I'm also sorry that I haven't been around for awhile. I'll use the usual and say I've been busy with Underground Gamer. Feel free to keep me busy and buy some games :D

If you need to get me ASAP then drop me a PM. I have it set to send me an e-mail.

-Joshua Estes

[tlinu]

Content visible to registered users only.

I am having same error as above. Anyone got a fix for that bug? Please share it...

Linu

[FB]

When you get it working, I'll give it another shot. for now I deleted all the files & database, as it just wouldn't work.

[f1gm3nt]

I don't know what to really tell you. It may take some time before I get another version out. The next link project that I am starting would have no need for this. I'll try to get another version out that is almost fully integrated into the phpLD. This should make it a little easier for everyone and they may include it in future versions.

Has anyone out there got the one for phpLD 2.0 working?

-Joshua Estes

[kodakball]

Content visible to registered users only.

Well i found a solution to fix this problem but, I didn't name the message properly my bad :lol:

Look a few messages back for my post and the fix is there.

or if you don't want to modify file yourself then you can download my modified version from my website
{non resolving url removed}

[Neticus]

Yes i've got it working but its highly tempermental. What i've found is...(And maybe this just my own badluck) but..

If you crawl within a category you may get the links but not the Title of the category you're in. This means that no category record is created in the 'category' SQL table which means convert.php won't convert your crawled links because it has nowhere to put it.

Take the category: Outplacement

If I click on the category Outplacement (http://dmoz.org/Regional/Europe/United_Kingdom/Business_and_Economy/Employment/Outplacement/) and crawl its URL

- The process will crawl all 9 links. But not category title Outplacement.

Without it recording a category title the links are useless because they have no category specified to go in.


I can only get it to crawl the category title by crawling the page before, that shows Outplacement as a category title:

thus (http://dmoz.org/Regional/Europe/United_Kingdom/Business_and_Economy/Employment/)

The only problem with this is the crawl will crawl all categories and links and not just the Outplacement category. Then i'll have the tedious task of going into database and deleting all the 300 links and categories i didnt want but the mod had to crawl on its way to crawling Outplacement. (aaarh!)

Once I work out how to manually insert Category details into the SQL Table then I'll be all right, right now i just can't seem to trick the system into converting when I do.

!

[f1gm3nt]

This is the way I've made it. I created it mostly to use on new directories to get them started. I'm not sure the exact wording for it but it crawls everything it can find with your prefix category. So for example give it the category

http://dmoz.org/Regional/Europe/United_Kingdom/Business_and_Economy/Employment/

and it crawls all the URLs that start with this. Does this make any sense? Now after you finish crawling this category it converts it by making all the next directory at the top of your directory. So given your example it will not import

http://dmoz.org/Regional/Europe/United_Kingdom/England/Business_and_Economy/Employment/

because it doesn't start with the prefix URL you gave it. The top categories would be like 'Outplacement'

If you are just wanting to import 9 links in the Outplacement category it would be more efficient to add them yourself. I hope this helps some people out.

-Joshua Estes

[rouseindahouse]

Mine is quite simply refusing to add links from any of the categories, just adding the categories themselves. I've sent you a PM about it but just in case you didn't see it, I thought I'd post as well.

[Neticus]

Content visible to registered users only.

That was just an example Outplacement could have 50 links which makes it less practical.

I know it may seem that people are being quite ungracious, I'm just pointing out a possible nuisance... I hope constructively...although im not sure. :roll:

It would be good if crawling within a specific category like Outplacement that you could set the category title manually in crawl.php if not needed then you could leave that area blank. The 'option' category title will automatically be picked up by crawl and inserted into the SQL table as the category for the links to go into before convert.

Thats just a thought thats all but really thanks for the mod it really is appreciated f1gm3nt :D

Cheers
Neticus

[f1gm3nt]

Content visible to registered users only.

All input I get from people I consider constructive. I really like your ideas, but right now I'm going to stick with getting it to use phpLD functions. After that I'll try some other options that you guys have asked for.

-Joshua

[f1gm3nt]

Content visible to registered users only.

The problem is the row count. Like the row in your database is 10 columns and the script is giving it 9 or 11 or some number other then 10. You can check your database tables and then double check the code. Let me know if there is a difference.

-Joshua

[rouseindahouse]

Sorry to sound like a <insert friendly insult there> but where exactly in the database and the code should I check. I'm doing all the database stuff in phpmyadmin.

[f1gm3nt]

Since you are having trouble with inserting links into the phpLD table look there. Then open up convert.php and check to make sure it adds all the correct data. If you don't know basic php/MySQL then I wouldn't really suggest it.

- Joshua

[paradox]

Content visible to registered users only.

Hi and thanks for this mod but I'm getting the same prob as FB.

The crawl just lasts for a second or two then when convert is activated - nothing :(

Is there a correction that I need to do? Does anyone have this working with the new version?

TIA for any help

[xxx]

Hi try this currently, Apear to work but when i run convert i got this ;

SQL ERROR:1146: Table 'liens.PLD_LINK_SEQ' doesn't exist

I have a table 'PLD_LINK' in my DB
but not 'liens.PLD_LINK_SEQ'

"liens" was the DB name in my case, but the DB name
is not in the table itself.. & i dont have a PLD_LINK_SEQ after
a fresh install of PHPlink..

How i can fix this ?
thanks.

[paradox]

Hi xxx

I had the same error. The "PLD_LINK_SEQ" is not created in the database until ay least one link is already there. Put in a link via admin and the table will appear for future links.

Do you have the DMOZ spider working OK with the current version?

[xxx]

Content visible to registered users only.

I have ended creating the PLD_LINK_SEQ manualy,
now it still importing currently, apear to work will see when this end..

I have make some try before then got some error whit convert.php
like this : (SQL ERROR:1136: Column count doesn't match value count at row 1) and i have seen on the DB that some link havent been converted for some reason..

Also about Dmoz, some link use special caracter ex: ;
http://dmoz.org/World/Fran%c3%a7ais/

All the directory name when importing in phplink display ; ( Fran%c3%a7ais ) in place of (Français) as exemple.. I dont know if this can be corrected...

I try importing http://dmoz.org/World/Fran%c3%a7ais/
180 000 link .. Running since 2 days now..

[f1gm3nt]

Yea, there is some problems with some characters. However here's how to fix this little problem. Open class.dmoz.php and find this code. It's near the top.

Content visible to registered users only.

To change the different characters to be different try that. So just add something like:

Content visible to registered users only.

If anyone knows of a better way to do this let me know.

-Joshua

[f1gm3nt]

For some reason I can't post a fix to the SQL error problem. Submitting it just takes me to the home page. I'll play with it more and see how good it works.

Sorry,
Joshua

[xxx]

hi,

After some try my suggest for improvement ;

1 - the SQL error came back for some reason...

SQL ERROR:1062: Duplicate entry '166077' for key 1
then all time i run i got this error now , i have not checked how mutch link stay in temp dir, will take a look..

2 - Another phpfile to erase entry in the temp
directory you use on the DB should be a nice addition
to erase all stuff on the fly on the table you use before convert...

3 - Add these to replace caracter;

$replace = array(
"_" => " ",
"%27" => "\'"
"%c3%a7" => "ç"
"%c3%a9" => "é"
"%c3%a8" => "è"
"%c3%b4" => "ô"
"%c3%a2" => "â"
"%c3%aa" => "ê"



if you can fix that that will make a 100% fonctionnal i think. :)

[ibold]

anyone else have an issue trying to index the entire dmoz site? Doesn't appear to recognize any categories on it.
Also, when running convert.php
Content visible to registered users only.

No dice. Tried a few of the suggestions listed here without luck. Any new idea's floating around out there? Thx in advance, and awesome script once I get it working :)

[Neticus]

Content visible to registered users only.

You may have inserted another mod that told you to add something to your database that changed the row count of default phpbb tables.

Quote from f1gm3nt on earlier same:
Content visible to registered users only.

I had the same problem and had to check if any mods I had installed had asked me to add something to my database especially the links and categories database. For me this mod was the Display URL Mod

http://www.phplinkdirectory.com/forum/viewtopic.php?t=1305

Since it changed some values i had to edit convert.php, in the below code my Pld Category Values now looks like this:

Content visible to registered users only.

And also in convert.php my phpLD Link Values after adjustment looks like this:

Content visible to registered users only.

After these adjustments the error of 'Column count doesn't match value count at row 1' stopped.

Other considerations:

I found that running convert.php from the link given after the crawl did not work. However it did work when I run /convert.php independently, by clicking the link in my file manager or by pasting the link in google and clicking from there.

Also I realised that if the category name already existed then Dmoz Mod would not create the category again and at the same time it will also not import the links that were supposed to come with that category. Maybe that's why some of you see categories but no links.

I fixed this by changing the name of the crawled dmoz category via database before running convert.php.

Before I import and convert a new category and links I have to delete all previous dmoz mod database table cats and links just to make sure it dosen't conflict with new crawl. Otherwise it will just import all the stored links from the previous crawl as well as the new crawl.

In a Category it will not import related categories i.e. categories with '@' prefixed. You have to go to that category itself to import the links.

Of course my database structure may be different to yours, however just a few observations for consideration in regards to future development or troubleshooting.


Net.

[paradox]

That info is very helpful. Thanks Net

[f1gm3nt]

Content visible to registered users only.

You forgot commas at the end of each entry. Put it like these

"_" => " ",
"%27" => "\'",
"%c3%a7" => "ç",
"%c3%a9" => "é",
"%c3%a8" => "è",
"%c3%b4" => "ô",
"%c3%a2" => "â",
"%c3%aa" => "ê"


Sorry I haven't been back in awhile things have gotten real crazy around here. I should have an update soon if time allows.

[Fred]

help me i am getting this aswell
Content visible to registered users only.but the only mods i have is
search mod and sugest mod
nonm of these affect the db
any ideas ??? :roll: :roll:

[Neticus]

Content visible to registered users only.

If your database has not been affected by any mods then perhaps try this:
http://www.phplinkdirectory.com/forum/viewtopic.php?p=7619#7619

[f1gm3nt]

Try downloading the new code found on the first page.

[buli]

i had problems with characters like ä ü ö A Ü Ö
so i added some things (class.dmoz.php) in function ReplaceChr($data)

Content visible to registered users only.

[hardcopi]

Just wanted to say GREAT MOD. I just started messing with my directory yesterday and here is a link to it so far:

I still have some cosmetic work to add to it as well as integrating it into my news aggregator software. Still it works incredibly well. I was able to use the parent_id field to import each category separately.

Thanks

[techbull]

Problem here...


I have added to fix caracter. ;

"_" => " ",
"%c3%ae" => "î",
"%c3%a7" => "ç",
"%c3%a8" => "è",
"è" => "è",
"é" => "é",
"û" => "û",
"ê" => "ê",
"â" => "â",
"î" => "î",
"%c3%a9" => "é",
"%27" => "\'"


That apear to work in description all fine but not in categories ??
All my categories that have been imported are not converted but all cat. description are fine ?

Ex: in my phplink after import all directory still have %c3%a9

Dir in dmoz ;
http://dmoz.org/World/Fran%c3%a7ais/R%c3%a9gional/Am%c3%a9rique/Canada/Qu%c3%a9bec/Actualit%c3%a9_et_m%c3%a9dias/T%c3%a9l%c3%a9vision/

After import in phplink 3.04 ;

www .mydir/T%c3%a9l%c3%a9vision/

Any way to fix that ??
What i do wrong here ?

[techbull]

Thanks for replying myself... :wink:

Thats a big to catch how those crapy code thing works...

i have changed whit this in class.dmoz.php ;



"_" => " ",
"%27" => "\'",
"è" => "è",
"é" => "é",
"û" => "û",
"ê" => "ê",
"â" => "â",
"î" => "î",
"ï" => "ï",
"%27" => "\'",
"%c3%aa" => "ê",
"%c3%ae" => "î",
"%c3%af" => "ï",
"%c3%a2" => "â",
"%c3%a4" => "&auml;",
"%c3%a8" => "è",
"%c3%a7" => "ç",
"%c3%a9" => "é",
"%c3%b4" => "ô",
"%c3%b6" => "&ouml;",
"%c3%bc" => "&uuml;",
"%c3%9c" => "&Uuml;",
"%c3%9f" => "&szlig;",
"%c3%84" => "&Auml;",
"%c3%96" => "&Ouml;"



f1gm3nt : Work fine now whit this added, the other mod dont work at all... They just load the server CPUto 90%+ then look to do nothing else.. At less i cant let the cpu like this for hours so... !

Thank-you

PS: this should be integrated into phplink, whit feature to select category 1 by 1 and a way to add the categories in sub category in phplink.. Thats what i think at less !

[hani]

Hello,

I use version 3.05 and get this error, I I click on:
Crawl complete! Now run covert.php

SQL ERROR:1136: Column count doesn't match value count at row 1

I use the standard clean installation without any mods.

Anyone can help me?

Best Regards,
hani

[hani]

Hello,

anyone can modify the script to run it with the new version 3.05 of phpld?
I would pay for.

Best regards,
hani

[neverknow1]

Content visible to registered users only.
Cool stuff........ will download and try that.........

[royden]

Hi,

Can someone point me to a place to download the latest version of this script or send me a copy.

The one mentioned above 'http://www.animepub.com/dmoz/Dmoz.zip' just goes to the homepage no download??

Thanks, Royden.

[Jeff]

Ditto the last poster - zip file in first post seems to be gone now :(

Is there a new dmoz import script available? I'm looking for something that will allow me to be selective, pick categories, and then manually choose sites from that list to include or delete as a starting point.

[web_design]

Hmm. it seems not work for me too. Any new updated ?

[royden]

Content visible to registered users only.
I think the Google to phpLD Import Mod (http://www.frozenminds.com/phpld.html) is more suitable for what you want to do? As you can not be selective with dmoz import script (apart from which directories to crawl) until you have finished the import (crawl & convert) and then remove categs and links from db or thru admin area. Just a thought.