Back
to indexFilippo Menczer
Summary
This paper identifies the inefficiency of running queries online, and the InfoSpiders algorithm is implemented in order to overcome this limitation. It is desirable to search with the most recent data available, but searching online is generally incomplete or inefficient. InfoSpiders addresses this problem through the use of distributed agents that adjust adaptively according to network bandwidth, and modify their population numbers based on the success of each agent. Agents learn through the use of evolutionary and reinforcement learning, with neural networks. These agents return their results to a primary server after a user-specific number of pages is reached.
This paper was extremely heartening to read, since it addresses one of the key concepts I would like to explore in my thesis: the personalizable nature of adaptive agents, as well as online searching and learning. Reinforcement learning using neural networks is a simple and elegant solution to this problem. In addition, the experimental evidence provided solidly indicates that this approach is worthy of future research.
Keywords
online search, scalability, coverage, recency, Q-learning, selective query expansion, word topology, linkage topology, relevance autocorrelation, reinforcement learning
Methods
The InfoSpiders algorithm effectively searches online through the use of distrubuted evolutionary agents that calculate the best link to follow based on a weighted keyword vector as input to a neural net. Training the neural net weights is accomplished through Q-learning, and reinforcement learning is used to reward the agent when a relevant page is found.
Rating
8
Bibtex Entry
@misc{ menczer99,
author = "Filippo Menczer",
title = "Is agent-based online search feasible",
text = "In Working Notes of the AAAI Spring Symposium on Intelligent Agents in Cyberspace, 1999.",
year = "1999",
url = "http://citeseer.nj.nec.com/context/1242128/0"
}