sábado, 14 de abril de 2012

Computer Science Classics and The Emperor’s Old Clothes

While I'm stuck in a middle of one of the hardest paper I've ever tried to read, I've decided to spend some time on reading Computer Science classics. The fact that I have a paper to submit, a dissertation to finish, dozens of other papers to read, and a long term project to work on seems to be the right motivation to procrastinate wisely.  While reading the last edition of the Communications of ACM, an article from Selma Tekir caught my attention to CS classics, which could be defined as those papers everyone wants to have read but nobody wants to read. Basically, she argues that classics, which are basically theoretical papers in CS written in the 50's and 60's are important to: 1) Providing fundamental theoretical knowledge on the field, 2) providing new perspectives and insights, 3) inspiring young professionals, 4) providing a common history to unite the community, and 5) facilitate the recognition of computer science as an independent science and profession.

The list of classics given by the author is the following:


  • “The Emperor’s Old Clothes,” C.A.R. Hoare
  • “An Axiomatic Basis for Computer Programming,” C.A.R. Hoare
  • “Gödel’s Undecidability Theorem,” S.F. Andrilli
  • “Computing Machinery and Intelligence,” A.M. Turing
  • “Reflections on Trusting Trust,” K. Thompson
  • “The Humble Programmer,” E.W. Dijkstra
  • “An Interview with Edsger W. Dijkstra,” P. Frana 
  • “Computer Programming as an Art,” D. Knuth
  • “The ‘Art’ of Being Donald Knuth,” E. Feigenbaum
  • “Donald Knuth: A Life’s Work Interrupted,” E. Feigenbaum



As an exercise, I've decided to read the “The Emperor’s Old Clothes,” by C.A.R. Hoare. Hoare is the inventor of Quicksort and is also a Turing Award recipient for his "fundamental contributions to the definition and design of programming languages", in 1980. This specific paper is basically a transcript of his speech at the Turing Award, when he described some of his experiences while working on several important project such as:

  • Quicksort
  • The first compiler for Algol 60
  • The Elliott 503 Mark II software system (a complete failure)
  • The formal definition of programming languages (Hoare logic and CSP)
  • The study of parallel programming and language constructs for operating systems (e.g., monitors)
  • Algol X (also a failure)
  • Algol W, which is based on Algol X
  • Algol 68 (Hoare considers it to be a failure, but some don't)
  • PL/I 
  • ADA, probably the most controversial topic of the text, about which he states: 
    • "Do not allow this language in its present state to be used in applications where reliability is critical, i.e., nuclear power stations, cruise missiles, early warning systems, anti-ballistic missile defense systems".

There are many interesting messages through the paper, but the strongest of them is about simplicity and elegance. Hoare argues that the lack of simplicity was the cause of the failure of the ambitious Elliot 503 system. In programming languages, these requirements are mainly due to the fact that the programmer should never be trusted, something already known in 1960. Another lesson is that a team of smart people working on a very challenging task and interesting problem can fail miserably due to the lack of focus, resources, documentation, communication, and control. Therefore, it was inevitable to remember some of my software engineering classes while reading the text.

After the Elliot 503 project, Hoare became an advocate of simplicity and elegance in any other project he participated. Although, he seemed to be right several times, I think he may have been too conservative. Some modern programming languages widely used today do not follow his standards. However, he makes an interesting point that nothing can stand against money. In other words, it is difficult to evaluate his points after the effort and money invested in some of the programming languages in use today.

Another conclusion from Hoare's speech is about how committees make so bad decisions in practice. This phenomenon is not new, but its reasons are still not clear to me. Hoare, together with the experts in programming languages at the time, has been part of several committees with the objective of defining a new programming language, and some of the resulting languages were a disaster.

An interesting point about Hoare's career is that he moved from industry to academia in order to work on the formal definition of programming languages, something that he was not able to do as business. This kind of transition still happens in Computer Science, but I think it is much rarer today. On the other hand, I've found several examples of people leaving academia to industry, specially to work for big companies such as Google, Facebook, and Microsoft.

Link: http://zoo.cs.yale.edu/classes/cs422/2011/bib/hoare81emperor.pdf

segunda-feira, 2 de abril de 2012

Sequential Influence Models in Social Networks

This paper is authored by some researchers from Cornell University, including Jon Kleinberg, and Yahoo! Research. It studies two influence models, one based on snapshots of a network and another based on detailed dynamics. In this models, vertices are activated w.r.t. communities and the metric of interest is how the number of active neighbors affects the probability of a node to be activated. The two models are compared and a technique for estimating the detailed temporal dynamics based on snapshot data is proposed and evaluated using a dataset from Wikipedia.

Previous work have studied social influence by measuring how the number of active neighbors of a node affects the likelihood of this node become active. In fact, this idea is the basis of a standard epidemic model called Linear Threshold Model. This paper consider two definitions of influence according to the network information available:

Snapshot definition: Based on information from snapshots of network at different points in time. Given the number of nodes with k infected neighbors at time t_1 (d_s(k)) and the number of nodes with k infected neighbors at time t_1 which are infected at time t_2 (n_s(k)), the metric of interest p_s(k) is defined as p_s(k) = n_s(k) / d_s(k). 

Ordinal-time definition: The complete information, which includes the link formation and the activation of each node along time is available. In this case, d_o(k) is the number of nodes exposed to k infected neighbors at some point in time and n_o(k) is the number of nodes that where k-exposed and became active before becoming k+1 exposed. The influence ration is defined as p_o(k) = n_o(k) / d_o(k).

These two definitions are evaluated using a dataset from Wikipedia. An undirected social network is defined based on links between pairs of editors for which at least one of them has written on the other's user-talk page. Users become active by editing an article. Therefore, each article has a community of editors that can spread over the network. A dataset that contains approximately 510K users of the English Wikipedia is used in the study. Moreover, for some analysis, results using the German and French versions of Wikipedia are also presented.

The first important result of the paper is that the influence definitions produce different results. In the following figure, the authors show the values of p_o, and p_s for different values of k and considering the ordinal-time and the snapshot definitions.


These results come direct from the values of n_o, d_o, n_s and n_d, which are shown in the following Figure. These functions seem to follow a power-law distribution.



Further, the authors study the relation between the two models based on the following sets:

  • B(t_1) = {(u,C,k_1) | u joined C before t_1 an u had k_1 neighbors in C at t_1}. These events do not affect p_s but do affect p_o. As a consequence, the values of  n_o and d_o are shifted upwards with respect to n_s and d_s.
  • J(t_1,t_2) = {(u,C,k_1,k_2)  | u had k_1 neighbors in C at t_1, u joined C between t_1 and t_2, u had k_2 neighbors in C at t_2}. These events contribute to both p_s and p_o. However, the contribution over p_o is stretched towards higher values of k.
  • N(t_2) = {(u,C,k_2) | u did not join C before t_2 and u had k_2 neighbors in C at t_2}. Such tuples contribute only to p_o.

The analysis of the slope and the y-intercepts of the fit of the functions n_o, d_o, n_s, and d_s to a power-law distribution shows the upward effect of B and N. The stretching effect of J is small.

The authors also present a very simple technique for simulating ordinal time data from snapshots. For the events in B, it is assumed that u joined C at a uniformly random time j between 0 and t_1. The same is done for J (u is assumed to have joined C at a uniformly random time j between t_1 and t_2). The events in N are not considered. It is shown that this technique works well in practice, producing values of p_o that are similar to those obtained from ordinal-time data. The accuracy of the technique improves with the number of snapshots and as the time between snapshots gets longer.

The problem studied in this paper is very specific, but it is certainly relevant. I missed more datasets in the evaluation. In several points, I wondered whether the results discussed were general enough. In particular, the arguments of the authors were too complex to me. I think these arguments should be better supported with more experimental results. Finally, although they argued their technique for simulating ordinal-time influence based on snapshots worked well, they do not present any quantitative measure to support it. Moreover, I did not see any reason why such a technique model is actually effective.

Link: http://www.cs.cornell.edu/home/kleinber/icwsm10-seq.pdf

terça-feira, 27 de março de 2012

Content-based Modeling and Prediction of Information Dissemination

This paper was written by two researchers from University of California, Santa Barbara, and is about the problem of predicting direct links that express communication using content information. It can be seen as a variation of the traditional link-prediction problem where extra information w.r.t. communication flows (aka threads) is available. The idea is to apply Latent Dirichlet Allocation (LDA) in order to learn topics based on content features and then use topic distribution to predict the graph structure.

Link prediction is a very interesting problem and the scenario presented in the paper is novel. To illustrate it better, lets consider a network extracted from email communication as example. In this network, each node is an address and links are messages sent. Therefore, the content of the message may be useful in the discovery of its list of recipients. In order to make use of such information, they propose the GC-Model (don't know what GC means). The content (words) of messages are represented as a "word document" and the list of possible subgraphs, with limited size, of the graph that describes the messages in thread as a "graph document". Therefore, each thread produces two documents. This nomenclature matches the concepts involved in LDA and other topic models.



The GC-model works as follows. Topic distributions are learned from content using LDA (I won't try to explain LDA here, but it is basically a generative model for the identification of topic, which are sets of words, based on documents). This is the content part. The probability of a graph word appearing in a document and the topic mixture of this document are related as follows:

P(word) = Sum_{i=1 to t} P(word | T_i).Q(T_i)

Where Q(T_i) is the topic probability in the given document and P(word | T_i) is defined using Bayes' theorem:

P(word | T_i) = P(T_i|word).P(word) / P(T_i) 

Where T_i is a topic. Each component of the formula is computed as follows:

Topic distribution: P(T_i) = Sum_{j=1 to n} P(T | Thread_j).P(Thread_j)

Thread probability: P(Thread_j) is computed based on the time between threads and their percentage of link overlap, which are binned and fitted to an exponential. Roughly, the probability of a thread structure occurring again decreases exponentially with the time since its most recent occurrence.

Word probability: P(word) = Sum_{j=1 to n} P(word|Thread_j).P(Thread_j)

Topic probability given the word: Estimated using sampling.

Based on this formulation, the probability of a graph word can be computed based on the thread topic distribution. Therefore, given training data (i.e., a set of training thread), this model can be used to predict communication structures in new threads based on their content.

The authors apply the proposed method to three real datasets: The Enron email network (threads are based on email subjects, 5k threads), and two Twitter datasets (threads defined based on replies and retweets, 60k and 400k threads). The length of graph words selected (empirically) is 2 and the evaluation metric used is F1 score. Baselines are standard metrics applied in link prediction (e.g., common neighbors), a naive technique that always predicts the most frequent link and other based on text distance, and the PropFlow (unsupervised and somewhat similar to PageRank) algorithm. The GC-model outperforms all the baseline methods.

The general idea of this paper is good, but I think it should be developed a little further. Regarding the presentation, I think it would be greatly improved by removing some pseudo-codes and adding more text to it. The experimental section is not very strong, specially because it seems like unfair comparing PropFlow and the other baselines against the GC-model, which is the only one that is supervised. Moreover, there is no much evidence that supports why GC-model is good.

Link: http://www.cs.ucsb.edu/~kpm/ASONAM2011_Macropol_Singh_final.pdf

quinta-feira, 22 de março de 2012

2012 ICWSM Conference: Accepted Papers

Here is a list of potentially interesting papers to be published in the proceedings of the ICWSM conference this year:

Learning the Nature of Information in Social Networks
Event Diffusion Patterns in Social Media
Virality and Susceptibility in Information Diffusions
Temporal Motifs Reveal the Dynamics of Editor Interactions in Wikipedia
Modeling Spread of Disease from Social Interactions
Modeling Destructive Group Dynamics in On-line Gaming Communities
What Were the Tweets About? Topical Associations between Public Events and Twitter Feeds
Cross-Community Influence in Discussion Fora
Have You Heard?: How Gossip Flows through Workplace Email
The Emergence of Conventions in Online Social Networks
Modeling Diffusion in Social Networks Using Network Properties
The YouTube Social Network
On the Study of Social Interactions in Twitter
Modeling Polarizing Topics: When Do Different Political Communities Respond Differently to the Same News?
SearchBuddies: Bringing Search Engines into the Conversation

Twitter is still hot (but for how long?). Many papers on social influence/propagation/diffusion. The fact that most of the papers are about online social networks is something kind of scaring, seems like an academic bubble.

Link: http://icwsm.org/2012/program/program/

terça-feira, 20 de março de 2012

Output Space Sampling for Graph Patterns

No doubt that VLDB has published some great papers recently. This one is certainly part of my top list. It was written by some colleagues from RPI. The problem studied is subgraph pattern mining. The authors propose an output sampling algorithm based on Metropolis-Hastings that is able to produce graph patterns according to a general purpose interestingness metric (e.g., support) and threshold (e.g., minimum  support). Moreover, you can also bias the sampling method towards a given distribution. The benefits are: (1) Performance improvement over traditional graph mining algorithms, and (2) reduction of the output volume. Graph mining, specially frequent graph mining, is a problem that is not well-solved in the literature (the input graphs considered are too small) and sampling seems to be an interesting solution for it. The work is theoretically sound and has potential to affect data mining significantly in the long term.

The (generic) subgraph mining problem consists of, given a graph database (multiset of graphs), and an interestingness function F->R, return all subgraphs from the database that satisfy the given threshold. The proposed method samples the output space of a subgraph mining algorithm without generating the whole search-space. This approach is completely different from what has been done in the literature so far.

I will skip the formal definition of what is a graph, a subgraph, isomorphism, etc. and go straight to the point. The idea of the output sampling is performing a random-walk on the search-space of subgraphs, which can be seen as a Partial Order Graph (POG). However, this random walk must be smart. This is when the Metropolis-Hastings (MH) algorithm arrives. MH enables the generation of samples following a given target distribution. MH is a Markov chain Monte Carlo (MCMC) method developed in 1970. One important characteristic of this method is that it depends only on the current state (Markov property).

MH applies an arbitrary jumping function Q, which, given a current value x_t, returns a new value x_{t+1}. Q must be symmetric, i.e. Q(x_{t+1} | x_t) = Q(x_t | x_{t+1}).  The procedure starts with an initial value x_0 and then enters into a loop where new values are generated based on the current value x_t and Q.  Given a = P(x_{t+1})/P(x_t), if a >= 1, then x_{t+1} is accepted, otherwise, x_{t+1} is accepted with probability a. P is the target distribution. The method works better if Q is similar to P. There are some conditions under which MH works, the authors consider the condition that P must be stationary (i.e., P will converge after some iterations in the Markov process).

The state space of the MH algorithm for sampling the output space of subgraph patterns is the Partial Order Graph (POG). This graph is build only locally and the authors prove that a random walk on POG converges to a stationary distribution (it is finite, irreducible, and aperiodic).

First, it is shown how to obtain a uniform sample of frequent patterns, what means that any frequent pattern has the same probability to be returned. In order to do so, it is necessary to create proper transition probabilities in the POG. In fact, they show that if the transition matrix is symmetric, then the stationary distribution is uniform. P(u,v), where u and v are states (or vertices in the POG) are defined as follows:

P(u,v)        =                          1/2(*max(d_u,d_v))                                       if u != v and v in adj(u)
                      3/2- Sum_{x in adj(u)} P(u,x)                        if u = v
                        0                                                                    otherwise

The MH algorithm is adapted using P(u,v). The algorithm receives a pattern (in practice an empty pattern, which is part of the POG), the minimum support threshold, and the number of iterations to be performed. The acceptance of a transition is based on the degree of the current vertex and the degree of the new vertex. The number of transitions depend on the number of iterations and the final pattern found is returned as output. In this process, only neighbors of the current pattern are generated (i.e., sub- and super-patterns) and the support counting for these patterns is performed using standard isomorphism checking in a pass over the database.

The authors show also how to perform a support proportional sampling (i.e., a sampling biased towards the most frequent subgraphs), which requires a modification of the transition function P(u,v). However, since they are still working with the POG, they must relate this interestingness function with the degree of its vertices. Basically, they propose a transition function where choosing an up movement (towards the empty pattern) is more likely than choosing a down movement in some extent. Changing the algorithm to consider a discriminatory metric is easy, since it requires only a small change in how new vertices are accepted.

The results are divided into two parts. First an analysis of the patterns using a small dataset (AIDS antiviral screening dataset) is presented. It is shown that the method generates a uniform distribution of patterns if desired and it converges after 20k iterations. The authors also show that the distribution of patterns is still a bit skewed towards high degree vertices in the POG. Using an transaction dataset, for which the POG is denser, they show that their method achieves a more uniform distribution. In terms of support-biased sampling, the method also works well, but it is necessary to select a good value for up/down transition probabilities. Discriminatory subgraph sampling is evaluated using a dataset of chemical graphs. The results in this case are not very conclusive. Although many high delta score patterns are found, the correlation between delta score and visit count does not seem as high as for the support-biased case. In the evaluation using large graphs, a protein-interaction and a cell-graph database are applied. Most of the existing algorithms take more than 2 days to process such datasets and the output space sampling algorithm is shown to be a good alternative.

The technique proposed in this paper is very innovative and relevant. In the light of what has been done so far in graph mining, this is a big step towards the analysis of real graph databases. One thing that I did not understand very well is why the approach proposed for support-biased sampling is different from the one that considered the significance of patterns. In fact, after finding out that the significance-biased technique does not work very well I start to wonder how general the technique is in practice. I would like to know the line of thought that brought the authors from the existing work to this sampling idea, it is really different from everything I've seen in this area.

Link: www.vldb.org/pvldb/2/vldb09-713.pdf

quinta-feira, 15 de março de 2012

PageRank: Standing on the shoulders of giants

This paper presents the PageRank algorithm from a historical perspective. PageRank is an algorithm for ranking web pages in terms of relevance based on the structure of the web graph. It was proposed by Sergey Brin and Larry Page in 1998 and has been applied by the Google search engine. This paper is authored by Massimo Fraceschet, from University of Udine, Italy, and was published by the Communications of the ACM magazine.

The HTML protocol, which is the basis for the Web as we know, was invented in 1989 by Berners-Lee while he was working at CERN. PageRank was one of the first efforts on the use of the web topology (that emerges from links between pages) to measure the importance of a page. The basic idea of PageRank is that a web page is important if it is pointed to by other important links. The formulation of the score is as follows:

PI = alpha.PI.S + (1-alpha).u

Where alpha is the damping factor, S is the adjacency matrix of the graph after dangling pages (i.e., without output links) have their output probabilities set as uniform, and u is a uniform probability vector.

The PageRank equation is known to have a unique solution. The existence of at least one solution comes from the fact that the transformed adjacency matrix is stochastic (i.e., all rows sum up to 1). Moreover, according to the Perron-Frobenius theorem,  the equation has a single solution.

Perron-Frobenius theorem: If A is irreducible (i.e., its associated directed graph is strongly connected) nonnegative square matrix, then there exists a unique vector x such that xA = rx, x > 0, and Sum_{i} x_i = 1, where r is the maximum eigenvalue of A in absolute value.

The solution to the PageRank equation can be obtained using the power method, which computes the eigenvector and eigenvalue of a matrix through an iterative process. An initial vector PI^{0} is set to u (uniform vector) and PI^{k+1} = PI^{k}. G, where G is the google matrix. The result converges after a limited number of iterations that depends on the damping factor alpha.

After presenting PageRank, the author discusses a list of very similar methods in Computer Science, Bibliometrics, Sociology and Economics. One of these methods is HITS (Hypertext Induced Topic Search), developed by Jon Kleinberg. HITS is based on the concept of authorities and hubs. Good authorities are pages that are pointed to by good hubs and good hubs are pages that point to good authorities. Given an adjacency matrix L, the authority (x) and hub (y) vectors are defined as follows:

x^k = L^T.y^{k-1}
y^k = L.x^k

These equations can be solved using the power method, like was discussed for PageRank.

I wont discuss the other methods in detail. The point is that ideas very close to PageRank and HITS were developed decades before in other areas.

Bibliometrics (1976): A journal is influential if it is cited by other influential journals.

Sociometry (1949): A person is prestigious if he is endorsed by prestigious people.

Econometrics (1941): Highly remunerated industries are those that receive substantial inputs from highly remunerated industries.

This is a great paper. It is funny how recurring ideas appear somehow independently in some different areas. In the case of PageRank and HITS, the research was developed almost in parallel. Something that I ask to myself sometimes is whether I should know Math a lot or just be able to model the a given problem and then search for the mathematical solution. I've heard somewhere that the second approach is better.

 Link: http://arxiv.org/abs/1002.2858

quarta-feira, 14 de março de 2012

Trust Transitivity in Social Networks

Here goes another paper written by Physicists. This paper studies trust in a social network where trust relationships are transitive. Understanding trust is a very relevant problem in society. In particular, several web applications, such as e-markets and P2P file systems, rely on trustful users in order to provide quality service. Trust transitive plays a special role in non-centralized scenarios, i.e. when there is not a central source of trust available.

Two trust metrics based on transitivity are proposed. The first one, called best trust, is defined as follows:

s_{u,v} = max{PROD_{e_i} c_{e_{i}}}, for all {e_i} in P{u=>v}

According to this metric, the trust of u towards v is the maximum trust product of the direct trusts over the path P{u=>v}is the set of all paths between u and v, {e_i} is the set of edges in a given path, and c_e is a value of direct trust associated to an edge. This metric can be computed using a shortest path algorithm, but the authors argue that it is biased towards high values of trust, so they propose another metric, called pervasive trust:

t_{u,v}=(Sum_w A_{w,v}.(s_{u,v}^{G\{v}})^2.c_{w,v})/(Sum_w A_{w,v}.s_{u,w}^{G\{v}})

Where A is the adjacency matrix, and s_{u,v}^{G\{v}} is the best trust from u to a neighbor of v (v is excluded). In this case, several paths are considered, instead of a single one. Moreover, this metric does not tend to 0 when N approaches to infinite.


The authors give a short discussion on the Eigentrust and the trustWebRank, which are other trust metrics in the literature. These metrics are based on the concept of feedback centrality and are calculated by solving some linear system. Eigentrust requires a stochastic matrix (i.e., the sum of the trust values of the out-edges of all vertices must sum to 1) and the inferred trust values are given by the steady state distribution of the corresponding Markov chain (i.e., the left eigenvector of the stochastic matrix must have an unity eigenvalue). Therefore, the trust values are not personalized and the absolute values of trust are lost. TrustWebRank is based on the PageRank algorithm and also requires a stochastic matrix but is personalized. It also depends on a damping factor that eliminates the contribution of longer paths.

Properties of the two trust metrics presented are studied theoretically. The authors consider a random direct network with arbitrary degree distribution and direct trust values between c and c+dc given by an arbitrary distribution. In this scenario <t> = <s><c> in the limit of large network size. The discussion seems to be heavily based on the paper "Random graphs with arbitrary degree distributions and their applications" that I've not read yet. Therefore, I will not cover the math here. Basically, the authors give a formulation for <s> and compute it through numerical simulations and an analytical solution. In- and out-degree distributions following a Poisson and a Zipf are studied. The authors found a first-order transition from vanishing to positive trust values that correspond to the formation of a giant component in the network when the degree distributions are Poisson. For a Zipf distribution, the transition is of second order and the average trust is smaller than for the Poisson distribution.

Trust transitivity is also studied using data from the PGP (Pretty Good Privacy) network, which is a mechanism for users to attach a signature to the public key of another user for cryptography purposes.  The largest connected component of this network has 40k vertices. It is shown that this network does not grow in a purely random fashion because the power-law distribution of the waiting time between new keys and signatures. Degree distributions seem to follow a power-law. Moreover, the in- and out-degree of  vertices are strongly correlated and the reciprocity is very high. The degree correlation is assortative for intermediary degree values and dissortative for very high and very low degrees. Two paradigms of trust organization are studied: Authority (trust relies on a few universally trusted vertices) vs. Community-based (densely connected components provide trust to each other). The authority-based trust is more efficient but also more prone to forgery. The community-based trust has opposite characteristics. These two mechanisms seem to work together in the PGP network. In order to identify the communities, the authors a modularity maximization algorithm is applied. Two types of communities were found: one that represents the vicinity of an authority vertex and another that represents a "real" community. As means to understand trust transitivity in the PGP network, the authors assign a direct trust c=1/2 to all edges except for a fraction gamma of edges to which c=1. Three scenarios are considered: 1) Edges are selected at random (Random); 2) Edges with the largest betweenness are selected (Authority-centered); and 3) Edges with largest edge clustering are chosen (Community-centered). The authority-centered trust leads to higher values of average trust (<s> and <t>). On the other hand, the community-centered trust leads to lower values of average trust.  Moreover, for random trust distributions, vertices with large degree achieve higher <s> but have uniformly distributed <t>, as desired. For authority-based trust, both small and large degree vertices achieve high trust values. For community-based trust, vertices with intermediary degree achieve higher trust than the small and large degree ones.

This paper is very interesting and relevant. It is similar to another Physics paper I've read some weeks ago. This pattern of studying complex features of simple models seem to be a standard for these kind of paper. In the case of this particular paper, a real dataset (the PGP network) is also analyzed. I found some minor typos that were probably corrected in the "official version" of the paper (this one is from ArXiv). I think that the formulation for <s> is not clear. I will read the paper "Random graphs with arbitrary degree distributions and their applications" in the near future in order to understand the formulation in detail.

Link: http://arxiv.org/abs/1012.1358