Academic literature on the topic 'Prefix search'

Made available in DSpace on 2017-07-21T14:19:39Z (GMT). No. of bitstreams: 1 Robson Fernando Duda.pdf: 3342961 bytes, checksum: 3f61d3a8f1dcfb461c6860c82d3f54db (MD5) Previous issue date: 2014-02-28<br>Coordenação de Aperfeiçoamento de Pessoal de Nível Superior<br>This paper presents a solution to the routing problem of vehicles with homogeneous fleet. To do so, heuristic and metaheuristic based algorithms applied towards the development of a decision support system, with georeferenced interface were developed. The algorithms had as base heuristic methods built in two phases, besides a metaheuristic. The interface layer used as visualization component is based in cartographic data that indicates the location of the points to be assisted and the paths that connects them, forming a road system represented using the Google Maps® API. The algorithms were validated using instances from the literature, presenting satisfactory results regarding optimization based in the methods that were used, showing that it is possible the usage of the developed system in the distribution of agricultural products.<br>Este trabalho apresenta uma solução para o problema de roteamento de veículos com frotas homogêneas. Para tanto, foram desenvolvidos algoritmos baseados em heurísticas e meta-heurísticas aplicadas ao desenvolvimento de um sistema de apoio a decisão, com interface georreferenciada. Os algoritmos tiveram como base métodos heurísticos construtivos e em duas fases, além de uma meta-heurística. A camada de interface utilizada como componente de visualização é baseada em dados cartográficos que indicam a localização dos pontos a serem atendidos e as vias que os interligam, formando a malha viária que é representada utilizando a API do Google Maps®. Os algoritmos foram validados utilizando instâncias da literatura, apresentando resultados satisfatórios em relação a otimização baseada nos métodos utilizados, mostrando ser possível a utilização do sistema desenvolvido para a distribuição de produtos agrícolas.

In this work I argue that field of text search has focused mostly on long text documents, but there is a growing need for efficient short text search, which has different user expectations. Due to this reduced data set size requirements different algorithmic techniques become more computationally affordable. The focus of this work is on approximate and prefix search and purely text based ranking methods, which are needed due to lower precision of text statistics on short text. A basic prototype search engine has been created using the researched techniques. Its capabilities were demonstrated on example search scenarios and the implementation was compared to two other open source systems representing currently recommended approaches for short text search problem. The results show feasibility of the implemented prototype regarding both user expectations and performance. Several options of future direction of the system are proposed.

Cette thèse étudie la problématique de l’indexation et de la recherche dans les tables de hachage distribuées –Distributed Hash Table (DHT). Elle propose un système de stockage distribué des résumés de documents en se basant sur leur contenu. Concrètement, la thèse utilise les Filtre de Blooms (FBs) pour représenter les résumés de documents et propose une méthode efficace d’insertion et de récupération des documents représentés par des FBs dans un index distribué sur une DHT. Le stockage basé sur contenu présente un double avantage, il permet de regrouper les documents similaires afin de les retrouver plus rapidement et en même temps, il permet de retrouver les documents en faisant des recherches par mots-clés en utilisant un FB. Cependant, la résolution d’une requête par mots-clés représentée par un filtre de Bloom constitue une opération complexe, il faut un mécanisme de localisation des filtres de Bloom de la descendance qui représentent des documents stockés dans la DHT. Ainsi, la thèse propose dans un deuxième temps, deux index de filtres de Bloom distribués sur des DHTs. Le premier système d’index proposé combine les principes d’indexation basée sur contenu et de listes inversées et répond à la problématique liée à la grande quantité de données stockée au niveau des index basés sur contenu. En effet, avec l’utilisation des filtres de Bloom de grande longueur, notre solution permet de stocker les documents sur un plus grand nombre de serveurs et de les indexer en utilisant moins d’espace. Ensuite, la thèse propose un deuxième système d’index qui supporte efficacement le traitement des requêtes de sur-ensembles (des requêtes par mots-clés) en utilisant un arbre de préfixes. Cette dernière solution exploite la distribution des données et propose une fonction de répartition paramétrable permettant d’indexer les documents avec un arbre binaire équilibré. De cette manière, les documents sont répartis efficacement sur les serveurs d’indexation. En outre, la thèse propose dans la troisième solution, une méthode efficace de localisation des documents contenant un ensemble de mots-clés donnés. Comparé aux solutions de même catégorie, cette dernière solution permet d’effectuer des recherches de sur-ensembles en un moindre coût et constitue est une base solide pour la recherche de sur-ensembles sur les systèmes d’index construits au-dessus des DHTs. Enfin, la thèse propose le prototype d’un système pair-à-pair pour l’indexation de contenus et la recherche par mots-clés. Ce prototype, prêt à être déployé dans un environnement réel, est expérimenté dans l’environnement de simulation peersim qui a permis de mesurer les performances théoriques des algorithmes développés tout au long de la thèse<br>This thesis examines the problem of indexing and searching in Distributed Hash Table (DHT). It provides a distributed system for storing document summaries based on their content. Concretely, the thesis uses Bloom filters (BF) to represent document summaries and proposes an efficient method for inserting and retrieving documents represented by BFs in an index distributed on a DHT. Content-based storage has a dual advantage. It allows to group similar documents together and to find and retrieve them more quickly at the same by using Bloom filters for keywords searches. However, processing a keyword query represented by a Bloom filter is a difficult operation and requires a mechanism to locate the Bloom filters that represent documents stored in the DHT. Thus, the thesis proposes in a second time, two Bloom filters indexes schemes distributed on DHT. The first proposed index system combines the principles of content-based indexing and inverted lists and addresses the issue of the large amount of data stored by content-based indexes. Indeed, by using Bloom filters with long length, this solution allows to store documents on a large number of servers and to index them using less space. Next, the thesis proposes a second index system that efficiently supports superset queries processing (keywords-queries) using a prefix tree. This solution exploits the distribution of the data and proposes a configurable distribution function that allow to index documents with a balanced binary tree. In this way, documents are distributed efficiently on indexing servers. In addition, the thesis proposes in the third solution, an efficient method for locating documents containing a set of keywords. Compared to solutions of the same category, the latter solution makes it possible to perform subset searches at a lower cost and can be considered as a solid foundation for supersets queries processing on over-dht index systems. Finally, the thesis proposes a prototype of a peer-to-peer system for indexing content and searching by keywords. This prototype, ready to be deployed in a real environment, is experimented with peersim that allowed to measure the theoretical performances of the algorithms developed throughout the thesis

A busca de palavras em uma grande coleção de documentos é um problema muito recorrente nos dias de hoje, como a própria utilização dos conhecidos \"motores de busca\" revela. Para que as buscas sejam realizadas em tempo que independa do tamanho da coleção, é necessário que a coleção seja indexada uma única vez. O tamanho destes índices é tipicamente linear no tamanho da coleção de documentos. A compressão de dados é outro recurso bastante utilizado para lidar com o tamanho sempre crescente da coleção de documentos. A intenção deste estudo é aliar a indexação utilizada nas buscas à compressão de dados, verificando alternativas às soluções já propostas e visando melhorias no tempo de resposta das buscas e no consumo de memória utilizada nos índices. A análise das estruturas de índice com os algoritmos de compressão mostra que arquivo invertido por blocos em conjuntos com compressão Huffman por palavras é uma ótima opção para sistemas com restrição de consumo de memória, pois proporciona acesso aleatório e busca comprimida. Neste trabalho também são propostas novas codificações livres de prefixo a fim de melhorar a compressão obtida e capaz de gerar códigos auto-sincronizados, ou seja, com acesso aleatório realmente viável. A vantagem destas novas codificações é que elas eliminam a necessidade de gerar a árvore de codificação Huffman através dos mapeamentos propostos, o que se traduz em economia de memória, codificação mais compacta e menor tempo de processamento. Os resultados obtidos mostram redução de 7% e 9% do tamanho dos arquivos comprimidos com tempos de compressão e descompressão melhores e menor consumo de memória.<br>Pattern matching over a big document collection is a very recurrent problem nowadays, as the growing use of the search engines reveal. In order to accomplish the search in a period of time independent from the collection size, it is necessary to index the collecion only one time. The index size is typically linear in the size of document collection. Data compression is another powerful resource to manage the ever growing size of the document collection. The objective in this assignment is to ally the indexed search to data compression, verifying alternatives to the current solutions, seeking improvement in search time and memory usage. The analysis on the index structures and compression algorithms indicates that joining the block inverted les with Huffman word-based compression is an interesting solution because it provides random access and compressed search. New prefix free codes are proposed in this assignment in order to enhance the compression and facilitate the generation of self-sinchronized codes, furthermore, with a truly viable random access. The advantage in this new codes is that they eliminate the need of generating the Huffman-code tree through the proposed mappings, which stands for economy of memory, compact encoding and shorter processing time. The results demonstrate gains of 7% and 9% in the compressed le size, with better compression and decompression times and lower memory consumption.

En los últimos años se ha desarollado mucho la literatura que utiliza modelos estocásticos de equilibrio económico general en economía abierta. En esta clase de modelos el primer capítulo estudia si el banco central tiene que fijarse en al inflación medida mirando al los precios al consumo (CPI) o a los precios a la producción. Se demonstra como la introducción de competencia monopolística en el mercado del trabajo y rigidez de los salarios nominales justifica el utilizo de la inflación medida sobre CPI. En el segundo capítulo el enfoque es sobre las diferentes volatilidades de la inflación entre paísos de la unión monetaria y como esto se puede relacionar con diferentes estructuras del mercado del trabajo. En el último capítulo se utiliza un modelo a dos paísos para estudiar las consecuencias de una subida del precio del petróleo sobre la inflación, los salarios reales y el producto interno bruto.<br>In these last years there has been an increasing literature developing DSGE Open Economy Models with market imperfections and nominal rigidities. It is the so called "New Open Economy Macroeconomics". Within this class of models the first chapter analyses the issue of whether the monetary authority should target Consumer Price Index (CPI) inflation or domestic inflation. It is shown that the introduction of monopolistic competition in the labour market and nominal wage rigidities rationalise CPI inflation targeting. In the second chapter we introduce matching and searching frictions in the labour market and relate different labour market structures across European countries with differences in the volatility of inflation across the same countries. In the last chapter we use a two-country model with oil in the production function and price and wage rigidities to relate movements in wage and price inflation, real wages and GDP growth rate to oil price changes.

The thesis consists of four chapters. The introductory chapter clarifies different notions of rationality used by economists and gives a summary of the remainder of the thesis. Chapter 2 proposes an explanation for the common empirical observation of the coexistence of infrequently-changing regular price ceilings and promotion-like price patterns. The results derive from enriching an otherwise standard, albeit stylized, general equilibrium model with two elements. First, the consumer-producer interaction is modeled in the spirit of the price dispersion literature, by introducing oligopolistic markets, consumer search costs and heterogeneity. Second, consumers are assumed to be boundedly-rational: In order to incorporate new information about the general price level, they have to incur a small cognitive cost. The decision whether to re-optimize or act according to the obsolete knowledge about prices is itself a result of optimization. It is shown that in this economy, individual retail prices are capped below the monopoly price, but are otherwise flexible. Moreover, they have the following three properties: 1) An individual price has a positive probability of being equal to the ceiling. 2) Prices have a tendency to fall below the ceiling and then be reset back to the cap value. 3) The ceiling remains constant for extended time intervals even when the mean rate of inflation is positive. Properties 1) and 2) can be associated with promotions and properties 1) and 3) imply the emergence of nominal price rigidity. The results do not rely on any type of direct costs of price adjustment. Instead, price stickiness derives from frictions on the consumers’ side of the market, in line with the results of several managerial surveys. It is shown that the developed theory, compared to the classic menu costs-based approach, does better in matching the stylized facts about the reaction of individual prices to inflation. In terms of quantitative assessment, the model, when calibrated to realistic parameter values, produces median price ceiling durations that match values reported in empirical studies.<p><p>The starting point of the essay in Chapter 3 is the observation that the baseline New-Keynesian model, which relies solely on the notion of infrequent price adjustment, cannot account for the observed degree of inflation sluggishness. Therefore, it is a common practice among macro- modelers to introduce an ad hoc additional source of persistence to their models, by assuming that price setters, when adjusting a price of their product, do not set it equal to its unobserved individual optimal level, but instead catch up with the optimal price only gradually. In the paper, a model of incomplete adjustment is built which allows for explicitly testing whether price-setters adjust to the shocks to the unobserved optimal price only gradually and, if so, measure the speed of the catching up process. According to the author, a similar test has not been performed before. It is found that new prices do not generally match their estimated optimal level. However, only in some sectors, e.g. for some industrial goods and services, prices adjust to this level gradually, which should add to the aggregate inflation sluggishness. In other sectors, particularly food, price-setters seem to overreact to shocks, with new prices overshooting the optimal level. These sectors are likely to contribute to decreasing the aggregate inflation sluggishness. Overall, these findings are consistent with the view that price-setters are boundedly-rational. However, they do not provide clear-cut support for the existence of an additional source of inflation persistence due to gradual individual price adjustment. Instead, they suggest that general equilibrium macroeconomic models may need to include at least two types of production sectors, characterized by a contrasting behavior of price-setters. An additional finding stemming from this work is that the idiosyncratic component of the optimal individual price is well approximated by a random walk. This is in line with the assumptions maintained in most of the theoretical literature. <p><p>Chapter 4 of the thesis has been co-authored by Julia Lendvai. In this paper a full-fledged production economy model with Kahneman and Tversky’s Prospect Theory features is constructed. The agents’ objective function is assumed to be a weighted sum of the usual utility over consumption and leisure and the utility over relative changes of agents’ wealth. It is also assumed that agents are loss-averse: They are more sensitive to wealth losses than to gains. Apart from the changes in the utility, the model is set-up in a standard Real Business Cycle framework. The authors study prices of stocks and risk-free bonds in this economy. Their work shows that under plausible parameterizations of the objective function, the model is able to explain a wide set of unconditional asset return moments, including the mean return on risk-free bonds, equity premium and the Sharpe Ratio. When the degree of loss aversion in the model is additionally assumed to be state-dependent, the model also produces countercyclical risk premia. This helps it match an array of conditional moments and in particular the predictability pattern of stock returns.<br>Doctorat en Sciences économiques et de gestion<br>info:eu-repo/semantics/nonPublished

碩士<br>國立成功大學<br>資訊工程學系<br>104<br>Packet classification is a key functionality of the Internet routers for many network applications, such as Quality of Service (QoS), security, monitoring, analysis, and network intrusion detection (NIDS). The existing well-known decision-tree-based schemes, like HiCuts, HyperCuts and EffiCuts, suffer from a memory explosion problem caused by rule duplication and the number of memory accesses are inseparable with the height of decision tree they built. In this thesis, we propose a scheme called binary search on buckets (BSOB) for multi-dimensional packet classification problem. BSOB performs binary searches on the ID (prefix) of ordered leaf nodes in the decision tree. We give a pointer to tree node pointing to its corresponding ancestor node based on their prefix. When searching to a certain node, the fast link can be utilized to directly access to its ancestor node that is related to the certain node. Hence, BSOB can improves the existing well-known decision-tree-based schemes that need to traverse the decision tree from the root node to some leaf nodes. In order to solve the memory explosion problem caused by decision-tree-based scheme, we divide the original rule table into multiple groups and construct the BSOB structure to each group. Moreover, we use node retain technique which retains the rule that causes duplication in internal node. Hence, the rule duplication problem can be reduced. For 12 types classifiers with 100,000 rules generated by ClassBench, our experimental results implemented on a PC software environment show that BSOB uses 2.5-2.8 MB memory and needs 8-15 memory accesses in average case and 21-51 memory accesses in worst case that is much better than the existing well-known decision-tree based schemes. We also show the update results of BSOB. Insert operation is about 1-3 times higher than search and the delete operation is close to the search speed. Because of the advantage of less memory usage and high classification speed, BSOB is able to address the packet classification problems in SDN environment that has more header fields and large rule sizes.

碩士<br>國立成功大學<br>資訊工程學系碩博士班<br>92<br>Fast packet classification is required for the increasing traffic demand because of the rapid growth of the Internet. Packet classification is often the first packet processing step in routers. Because of the complexity of the matching algorithms, packet classification is often a bottleneck in the performance of the network infrastructure. Most of the algorithmic solutions don’t scale very well. In this thesis, we proposed a novel packet classification algorithm based on the binary prefix search. The data structure of a d-dimensional rule table is converted to a d-level sorted array for binary search on each level. We evaluated our scheme by a variety of filter tables and compared it with the other existing schemes. Our experiments showed that the proposed scheme performs better than other existing schemes in terms of speed and storage requirements. Specifically, the performance improvements of the proposed scheme in classification speed over the aggregated bit vector are 29-97% and 63-75% for tables of 1K-20K 2D rules and 100-2000 5D, respectively.

碩士<br>國立臺灣大學<br>資訊管理研究所<br>92<br>According to the Moore's law and fast evolutions of PC components, the capability or capacity doubles about every two years, and people get more resources than the regular engaged. To stimulate these to better potential, Napster guides its members to share storages and to search in the joint jukebox while others, such as SETI, strike hard problems with great help from the Internet users. A scalable and efficient search structure is necessary to organize such resources in environments akin to the Internet. Distributed hash tables meet all of our requirements but support only to exact matchings since object names are sacrificed during search. For wider applications based on overlay networks, we want broader abilities to search of correlated objects given one key. We propose a list-based overlay network, called Treevial, with tree competence of search (prefix/range query) while storage load remains fair with two schemes. First, a simple but powerful distributed storage scheme gives equal shares to nodes by transferring load between neighbors of the basic list. Second, a search structure based on the concept from skip graphs (membership vectors) retrieves any targets by logarithmic messages and fetches successive objects in the original namespace with constant costs. We think our work provide efficient search operations but depends heavily on locality issues. One that gives good locality for our design is to introduce landmarks during the join process of nodes. We propose a mechanism from available nodes in the system rather than having specific cites known to every node since a self-organized structure adapts better to the Internet. This makes nodes to be near in overlays if they are close in underlying networks, and improves the original design to be efficient in maintenance and search. There are performance measures of our design. As the results from simulations, the measures about the costs of storage balancing and another about routing stretch reduce more than half after we give locality awareness. This work is adequate in range-query-intensive environments and provides efficient search operations and good maintenance costs by locality.