Welcome Visitor, Please Login or Register Now Wednesday, February 05, 2025 07:50 AM 
Forums Index > Mewsoft Products > Searchawy > spidering query
New Topic   New Reply
Search for:
Author Message
gofindit4me gofindit4me's personal page
Registered Member
Posts: 24 Display member's posts
Joined: 02-21-06 03:51 PM
Member Offline
View Member's Profile Send private message
 
Back to top
spidering query Edit Delete Reply with quote Quote
First Post Posted on: 10-23-11 05:14 PM next post first post
Hi\ \ I have searchawy and dmoz crawler installed.\ \ Is there another way of re-spidering all the links in the database to make sure they are all live and to update the site's keywords? When you retrieve sites from Dmoz they just have very basic descriptions from the Dmoz category and so are not so easily found when searching from my site.\ \ Thanks\ \ Mark
mewsoft mewsoft's personal page
Administrator Team Member
Posts: 5381 Display member's posts
Joined: 02-05-25 07:50 AM
Location: knxoville, TN, USA
Member Offline
View Member's Profile Visit member's website Mewsoft Corporation Send private message ICQ Messenger: ICQ AIM Messenger: AIM YIM Messenger: Yahoo MSN Messenger: MSN
 
Back to top
Re: spidering query Edit Delete Reply with quote Quote
Reply #: 1 Posted on: 10-23-11 06:10 PM next post previous post
No, this can not be done, you can not respider sites because if your users have accounts and have entered their own keywords for bidding these will be lost. We selected dmoz specially because it is very well reviewed my thounsands of editors which you or anyone else can not afford even google uses dmoz and almost all search engine useres dmoz for their database.

Mewsoft Support
www.mewsoft.com
gofindit4me gofindit4me's personal page
Registered Member
Posts: 24 Display member's posts
Joined: 02-21-06 03:51 PM
Member Offline
View Member's Profile Send private message
 
Back to top
re spidering Edit Delete Reply with quote Quote
Reply #: 2 Posted on: 10-23-11 09:19 PM next post previous post
OK. Then is there a way with this search engine of actually using the data retrieved by the DMOZ crawler? because most results come up without any keywords that allow the sites to be found using the search. Some dont even have a brief description.\ Each link needs to be spidered to get all of the relevant keywords for it to be found.\ With the cost per click advertisers, doesnt their position depend on the amount they bid for those keywords? so other sites having similar keywords wouldnt matter?
mewsoft mewsoft's personal page
Administrator Team Member
Posts: 5381 Display member's posts
Joined: 02-05-25 07:50 AM
Location: knxoville, TN, USA
Member Offline
View Member's Profile Visit member's website Mewsoft Corporation Send private message ICQ Messenger: ICQ AIM Messenger: AIM YIM Messenger: Yahoo MSN Messenger: MSN
 
Back to top
Re: spidering query Edit Delete Reply with quote Quote
Reply #: 3 Posted on: 10-23-11 11:25 PM next post previous post

It seems you are talking about the web spider not the dmoz plugin, the dmoz plugin will get the links and their description as you see them on the dmoz directory, the Web spider if you used it it can get links yes without description if it did not find metag tags or some header text on the spidered websites.

As for indexing, these options you set in your admin center, you can set the engine to index sites Title, Desciption, Keywords, ifyou disabled these then it will not index any.


Mewsoft Support
www.mewsoft.com
gofindit4me gofindit4me's personal page
Registered Member
Posts: 24 Display member's posts
Joined: 02-21-06 03:51 PM
Member Offline
View Member's Profile Send private message
 
Back to top
spidering Edit Delete Reply with quote Quote
Reply #: 4 Posted on: 10-24-11 01:53 PM next post previous post
Right I see. That must be where I am going wrong.\ \ It is a brilliant script. The site I am building is looking great and offers bags more than my old one. Its just a bit complicated for a novice like me to work out - so many feattures!!\ \ So in the admin section I can set it to search descriptions, keywords and meta tags?\ \ I'll go and have another play ..\ \ Thanks
gofindit4me gofindit4me's personal page
Registered Member
Posts: 24 Display member's posts
Joined: 02-21-06 03:51 PM
Member Offline
View Member's Profile Send private message
 
Back to top
Re: spidering query Edit Delete Reply with quote Quote
Reply #: 5 Posted on: 10-24-11 02:07 PM next post previous post
OK, Still not got it sorted yet. I have selected search by title, description, keywords etc but still only seems to be going by title and description. I must be missing something.\ \ Another quick question. I cant find the answer in your documentation pages.\ \ in the directory manager there is a featured + and featured ++ option for the listings. What is this for??\ \ Also I have added a DJ's website to my directory. For some reason the search will find every other word he mentions in his description ecxept for DJ, it completely ignores it even though its not on my ignored words list. Any ideas? That is a major search keyword for him.\ \ regards\ \ Mark
gofindit4me gofindit4me's personal page
Registered Member
Posts: 24 Display member's posts
Joined: 02-21-06 03:51 PM
Member Offline
View Member's Profile Send private message
 
Back to top
Re: spidering query Edit Delete Reply with quote Quote
Reply #: 6 Posted on: 10-24-11 05:25 PM last post previous post
Ive spent a bit of time on this now and still cant get it to do what I want. Also the search methods listed in the 'tips' link dont seem to make any difference to the search results.\ \ Any suggestions?\ \ Thanks
Page 1 of 1
Go to page:

New Topic   New Reply Mark Unread
Jump to:  
Delete   Move     Lock   +Favorits   +Notify   Print  

  Mewsoft and Mewsoft Logos are registered trademarks. ® Corporation. All Rights Reserved.  Copyright -  Privacy -  Terms of Use  Page Load: 0.3759 Seconds 
Powered by Boardawy © 2000-2011. Mewsoft Corporation