Simple question: does the new Simple site search action in Freeway find content added to a site with Webyep?
regards
Marcel
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Doubtful. WebYep runs on the server, and stores its content there. SSS
runs in Freeway, and uses the Freeway publish event to update its
JavaScript database. WebYep could be extended (somehow) to add its
content to the uploaded SSS database, and to notice when Freeway had
overwritten that extended database with a new version (missing the WY
content) but that would be where it would have to happen, and it would
be a slight logistical nightmare, mostly to do with file permissions
on the database.
Walter
On Sep 24, 2010, at 10:01 AM, dadooper wrote:
Simple question: does the new Simple site search action in Freeway
find content added to a site with Webyep?
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
I need to address this issue as well. Any suggestions about how to do a search using webyep? I found something in the webyep forum, http://forums.obdev.at/viewtopic.php?f=6&t=981&p=10568&hilit=search#p10568 but it just made suggestions to search engines.
They also suggest, “So the most important point is to use a search script/engine that does not read the files of the website but instead access the website via HTTP, like an ordinary visitor does. It can do this from the host, your website is running on, but also from some other host.”
Anyone actually done this and have any experience with it?
Thanks every so much.
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
There’s a great Open Source search engine called Sphider, if you
search this forum, I believe one or more Freeway users have integrated
it into their sites. It uses a spider application coded in PHP (hence
the name) to amass a database of search terms and their matching
pages, so it could do what the WebYeppers are talking about here. It’s
not exactly simple to set up, but once you have it going, you would
simply visit a link to get it to re-index the site once in a while,
and you would be able to have integrated static/dynamic content in
your search results.
Walter
On Oct 12, 2010, at 3:33 PM, jan smoot wrote:
I need to address this issue as well. Any suggestions about how to
do a search using webyep? I found something in the webyep forum, http://forums.obdev.at/viewtopic.php?f=6&t=981&p=10568&hilit=search#p10568
but it just made suggestions to search engines.They also suggest, “So the most important point is to use a search
script/engine that does not read the files of the website but
instead access the website via HTTP, like an ordinary visitor does.
It can do this from the host, your website is running on, but also
from some other host.”Anyone actually done this and have any experience with it?
Thanks every so much.
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Has anyone used this WebYep/Sphider/Freeway combo?
I see Sphider has a custom ability to index from the command line. Is that what you are suggesting? Build the webyep pages, then process them a page-at-the -time kinda thing?
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Well first:
Yes me and of course David Owen. Realated to the list http://www.freewaytalk.net/thread/view/68151#m_70169 I carefully followed David’s and Walter’s words to achive this. The list includes the possibilities and needs of a search facility.
My example:
http://www.kimmich-dm.de/suche/index.php
David:
The indexing is manual. So if you add new content you need to index again.
Thomas
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
If you combine the command-line tools wget and cron, you can automate
this. The search engine on FreewayTalk (Sphinx) is re-indexed every
five minutes using a similar combination. Or, I imagine you could add
a hook to WebYep to re-index every time it saves.
Walter
On Oct 12, 2010, at 6:15 PM, Thomas Kimmich wrote:
The indexing is manual. So if you add new content you need to index
again.
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Thomas,
I could not make heads or tails out of the discussion at the thread you sent. There is mention of emailing an action - but no mention of what the action name is or how to get it.
And, I still don’t at all get how this works with Freeway and Webyep. Sphider talks about php and mysql. Their first instruction is to create a mysql database. How do I do that with webyep?
I am very lost. Should I download Spider and follow their installation instructions? Then do I follow their instructions about Indexing from the Command Line?
And, what is wget and cron? Do I need my host provider to support something specific?
Thanks for your replies.
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Sphider talks about php and mysql. Their first instruction is to
create a mysql database. How do I do that with webyep?
As far as this part is concerned you could use your hosting control
panel to create the database (there are also applications that could
accomplish the same thing such as Sequel Pro, for example). In any
case this part is a completely different animal from WebYep. Apple and
oranges as it were.
Todd
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Agree Todd, different steps to do. It’s a complex theme and believe me, I was sometimes at a point where I thought Sh***. It took me days to understand and it took Walter and David time and nerves as well to point me through.
You don’t have to do anything in or with webyep. The steps are:
MySQL: Create a database using your “provider control panel” or asking provider how to do.
Sphider: edit database.php (using editor like textedit) in the settings folder and fill in following lines:
$database=“Name of your Database”;
$mysql_user = “Username”;
$mysql_password = “Userpass”;
$mysql_table_prefix = “for example sphider”;
Line 1-3 you should get from Providers controlpanel or from provider directly, 4 I named “sphider_”.
Rename sphider-folder to a name you like, for example “search” and upload it to your root.
-
Open www.yourdomain/search/admin/install.php script (admin directory) in your browser, which will create the tables necessary for Sphider to operate.
-
Open www.yourdomain/search/admin/auth.php to change the administrator user name and password (default values are ‘admin’ and ‘admin’).
-
Open admin/admin.php in browser and start indexing.
At this point, the mentioned list started, because I recognized, that my server didn’t want to be crawled by sphider. The funny thing: After waiting about a night, he did. On other servers I haven’t had this problem.
Try to proceed up to here. After this I suggest to go on in http://www.freewaytalk.net/thread/view/68151#m_70169 cause one general discussion is enough and helpful for later users to proceed.
The mentioned action is just a small but very helpful one to prevent parts of your Freeway-Page from indexing by sphider (footer-area…). But this is the very very last step and you can get it from me.
Hope this helps you out for the moment.
Thomas
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Thomas,
Thanks so much for your reply. It seem that what you write about does not take WebYep into account. We are using WebYep and want to index a WebYep site. So, not sure I understand your first comment that “You don’t have to do anything in or with webyep.”
For your strategy to work - don’t I have to have my webyep data in a mysql database?
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Actually, what he is saying is that it does work for WebYep or any
other “virtual content” scheme. Sphider stands outside of your site –
it doesn’t even need to run on the same server that it is indexing at
all.
It behaves just as the Google-bot or any other “web crawler” does. It
reads the source of the first page on your site, just as if it was a
normal visitor using a browser, and it saves a precis of the page
content and a list of keywords in its own database. Then it looks
through the list of links on that page, picks the first one, and
follows it to the next page.
At each page, it adds to the list of links to follow, and to the
database of keywords and page descriptions. It is a recursive system,
so it can start anywhere and will eventually get the the bottom of
your site. You could also configure it to crawl and index another site
– even one you do not own or administer – if that’s what you’re
interested in doing.
Once this index is amassed, it can be used by the Sphider search form
on your site to produce a list of search results. Again, those results
may be from virtual pages or static HTML pages on your server, or even
pages on another server. Sphider doesn’t care one bit either way. The
only requirement is that you have PHP/MySQL on your server so you can
run the indexing system and the search form. The content format and
location are irrelevant.
Walter
On Oct 13, 2010, at 9:38 AM, jan smoot wrote:
Thomas,
Thanks so much for your reply. It seem that what you write about
does not take WebYep into account. We are using WebYep and want to
index a WebYep site. So, not sure I understand your first comment
that “You don’t have to do anything in or with webyep.”For your strategy to work - don’t I have to have my webyep data in a
mysql database?Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Walter,
Got it! I am evaluating Sphider and Zoom right now. Thanks for the push in the right direction. All for now.
Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Look:
it’s a three steps work:
-
MySQL Database
-
Settings within Sphider
-
Implementing Sphider into Freeway-Page.
What I described was Step 1 + 2. These two steps let sphider crawl your entire pages (including the data-files in webyep-system), list the keywords into the MySQL-Database (from Step 1).
The final (and for me the hardest) thing is to implement this into your freeway-artwork and this is listed in the other thread.
So just start with the steps 1+2 that I described in the previous message. And don’t panic - if you need a search facility that shall handle WebYep Content - this is a possible way to achieve. I’m more than underaverage in “scripting” and finally I got a result.
Thomas
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Sorry Walter and Jan, I was too lame (as ever), so keep on going 
Thomas
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Hi jan I mentioned this on the webyep forum but if you have seen it the alternative is
Wrensoft’s zoom application …and here is the userguide
http://www.wrensoft.com/zoom/usersguide.html
generally you need to populate the site first and then you use wrensofts own software (pc only although I used it via parallels) to scan and catalogue the site for all the words used… then it generates a document which contains all the text and finally this is uploaded via ftp to your site were the search facility (which I copied from wren soft) is able to view and use it
you need to do this at regular intervals if your site changes a lot.
Here is a working example where I have used it in conjunction with web yep
http://www.bestcustomsbrokercourse.com/searchpage.php
if you type in something like test in the search field it will find the relevant words on each page
all the best max
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Thanks Max,
I look forward to implementing this. Will be in our next phase. Grateful there is a way to accomplish what they want. Will be a month or so. I will check Zoom out. Seems simpler.
Thank you everyone. Jan
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options