======= Review 1 ======= > *** Paper Summary: Please summarize the paper in your own words. The paper does a detailed analysis of spatio-temporal characteristics of mobile phone data from wireless providers, with the goals of identifying temporal and land use patterns. The authors use Exploratory Factor Analysis (EFA), which is a fairly standard methodology in statistical analysis. The authors use data from two providers for two cities and find interesting patterns. The paper is reasonably well written and is easy to follow. This has been an active area of research in many different communities, including data mining, statistics and networking. The related work discussed here is very partial, and similar general conclusions have been observed in other papers as well. So the specific novel aspects of the paper are not too clear. > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. + Important topic, presentation is reasonable + The statistical analysis seems sound and is discussed well > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. - There has been a lot of work on this topic. See, e.g., recent proceedings of the UrbComp workshop, along with the standard data mining conferences such as KDD and ICDM - The paper is completely on statistical analysis, and there is no mathematical analysis of any models, etc. > *** Quality of Writing: What is the presentation quality of this paper? A paper not well-written is not good for INFOCOM reputation and will have difficulty in attracting citations. Your overall rating should take this into consideration. > *** Additional Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). borderline - top 50% of all papers assigned to me for review, but not top 20% (3) ======= Review 2 ======= > *** Paper Summary: Please summarize the paper in your own words. The paper proposes using Exploratory Factor Analysis (EFA) to classify mobile network traffic records both temporally and spatially. By applying EFA to two data sets, the paper shows that EFA reveals interesting temporal and spatial patterns and outperforms state of the art approaches. > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The paper shows that a well understood method in feature analysis can be leveraged for solving the difficult problem of classifying mobile traffic demands both temporally and spatially. The paper is timely and ties to several ongoing efforts that aim to develop methods to analyze data collected from mobile networks both temporally and spatially to inform many fields that range from public transit to city planning. > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The paper does not show how the presented spatial and temporal classification can be used by network operators. In other words, there is a need to sketch a path for going from the presented graphical illustrations to simple metric(s) that network operators can act on. The paper does not offer clear guidelines for how one needs to verify and prepare her data set before applying EFA. Sec IV,B starts a discussion around this but it remains mostly descriptive with little insight about the values used for tuning EFA. > *** Quality of Writing: What is the presentation quality of this paper? A paper not well-written is not good for INFOCOM reputation and will have difficulty in attracting citations. Your overall rating should take this into consideration. The paper is well written. > *** Additional Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. The paper uses considerable space to describe EFA fundamentals. In my opinion this is a good thing to do, but it does not deserve two pages. Most of the explained mathematics is not referred to neither implicitly nor explicitly in the later sections. I would recommend shortening sec III and have less focus on the math. While I find the presented results interesting, I believe that the authors have barely scratched the surface. For instance, I was expecting more thorough analysis of the performance of EFA, its robustness to missing data and fluctuations in traffic. It is not clear how to assess the fitness of the identified key factors. When analyzing the TIM-2013 dataset, the paper looked at hourly aggregates for the whole time period and in a representation called the median-week. The data set, however, can be analyzed in much greater depth. For instance, a multi-scale analysis e.g. 10 minutes, 30 minutes, and even several hours can potentially reveal interesting patterns. In sec V.B, what causes the unique behavior during weekend afternoons? Minor comments (typos): - Each variable consists in the mobile traffic -> Each variable consists the mobile traffic - EFA does not only identifies -> EFA does not only identify > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). accept - top 20% of all papers assigned to me for review (4) ======= Review 3 ======= > *** Paper Summary: Please summarize the paper in your own words. In this paper the authors propose to use Exploratory Factor Analysis (EFA) to characterize mobile network activity in time and space. The authors use EFA because it allows characterization of spatial and temporal network activity simultaneously by identifying latent correlations between variables in the data. The authors use two datasets namely; TIM-2013 and Orange-2014 datasets for evaluation purposes. TIM-2013 dataset is used to evaluate network activity profiling and Orange-2014 dataset is used for evaluating land use deteciton. Results for network activity profiling show that in comparison to the state of art (Reference 16) more descriptive temporal profiles can be obtained using EFA. The authors also relate the temporal profiles to geographical regions. Results for land use detection show that the spatial profiles obtained using EFA are very similar to profiles obtained through previous clustering based approaches (Reference 20). The spatial profiles are further explained by relating each factor (land use class) to different temporal activity patterns. > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. - The paper deals with a valid, well known real world problem of spatio-temporal characterizing mobile traffic. - EFA gives improvement over state of art. Using EFA, the results are better for characterizing temporal network activity profiles. For spatial profiles, comparison with previous approaches show that EFA factors result in similar profiles as obtained from clustering based approaches. - Results are interesting and show improvements over previous approaches. - The authors also justify their choice of using EFA over Principal Component Analysis(PCA). Tests are performed to check the suitability of EFA for the input data. > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The evaluation methodology is missing some details. -In Figure 6, only two factors are discussed. Apart from the initial selection done through parallel analysis (PA), the authors should add discussion on how they have ranked different factors? Are some factors more important than others? For example in Figure 6, why factors 1-8 and 10 -15 were excluded. Why are they less important? I understand that EFA has some limitations and factors are not always useful, but it would be beneficial if more details are given on the selection process. -Results in Figure 7 show that factor shown in 7(a) is similar to 4 clusters. By combining these clusters do we lose any information? I understand that factor 1 represents residential users. How about the four clusters, do these clusters further categorize residential users? Can we obtain this breakdown if more factors are analyzed? -Figure 8 relates factors in figure 7 to temporal profiles. It would be useful to show corresponding temporal activity profiles of all the geographical regions covered by the cluster in Figure 7(e-h). -Preliminaries discussed in section III can be improved by providing examples related to mobile traffic data. -Adding more details about dataset will be useful. How was Table 1 created? Are the labels in the Table available in the dataset or generated manually? -Others: Term Classification is misleading, use characterization instead. Section II: "classify them via clustering" -> change to characterize or profile. Section V-B para before Takeaways: "is less rich than than returned" Footnote 4 should be moved to main body of the paper. > *** Quality of Writing: What is the presentation quality of this paper? A paper not well-written is not good for INFOCOM reputation and will have difficulty in attracting citations. Your overall rating should take this into consideration. The paper is well written and properly formatted. Figures are interesting and easy to interpret. The writing should be revised once to remove the typographical errors. > *** Additional Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). accept - top 20% of all papers assigned to me for review (4) #################################################################################################################### ########################### Reviewer comments for previous version submitted at MobiHoc 2016 ##################### #################################################################################################################### ======= Review 1 ======= > *** Paper Summary: Please summarize the paper in your own words. The paper studies network activity profiling and land use classification using aggregated mobile traffic dataset (i.e., hourly traffic volume in base stations in a city over a period of a few weeks). > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. - The authors apply the exploratory factor analysis techniques to the proposed traffic profiling and land use classification problems. - The paper has a clear description of the used exploratory factor analysis method. - The related work discussion of the paper is clear, concise, and insightful. - The insights obtained are clearly articulated (some new and some not). - The EFA technique is applied to two sets of mobile data (Milan and Paris). One state-of-the-art methodology is implemented for comparision. > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. - Principle component analysis is arguably a very close related methodology. I am wondering whether the ETA can outperform PCA, and if so, why. - It feels a little bit thin in the sense that one known machine learning technique is applied to a relatively well studied traffic classification problem. > *** Detailed Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. - The authors apply the exploratory factor analysis techniques to the proposed traffic profiling and land use classification problems.This is a suitable application of the technique. The characteristics of the technique and problem setting fit each other. The numerical evaluation also justifies the application of the technique. - The paper has a clear description of the used exploratory factor analysis method.The paper discussed the basic variable, the formulation, the application, and numerical implementation. - The related work discussion of the paper is clear, concise, and insightful. I find it interesting to read the discussion. - The insights obtained are clearly articulated (some new and some not). Each part has a takeaways section that summarizes the main messages. Most of the messages are not new or superising, but still interesting to read. - The EFA technique is applied to two sets of mobile data (Milan and Paris). One state-of-the-art methodology is implemented for comparision. I would hope that PCA method could be compared. - Principle component analysis is argueably a very close related methodology. I am wondering whether the ETA can outperform PCA, and if so, why. The authors briefly discuss the difference between ETA with PCA in footnote 3. It would be better if 1) there are more detailed discussion on why differentiating the shared and unique variances matters in this context; and 2) a comparison can performed in the evaluation section, in addition to the current evaluation. I understand that space might be an issue; and if so, a discussion of the numerical comparison would be helpful. - It feels a little bit thin in the sense that one known machine learning technique is applied to a relatively well studied traffic classification problem. On the other hand, the authors did a very good job intergrating the technique with the problem. Also, the paper shows a clear understanding of the state-of-the-art with solid comparison. Both somewhat mitigate the concern of contribution. However, I probably would not be a champion for the paper. On one hand, I do not mind the paper being accepted - this method is relatively new to the audience of the Mobihoc community. On the other hand, it is a relatively standard application of the method. Furthermore, the problem of traffic profiling and land usage classification is relatively well-studied. So the paper can go either way. > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). Likely Reject (This paper should be rejected but I will not fight strongly against it) (2) Reviewer familiarity - Good knowledge (3) ======= Review 2 ======= > *** Paper Summary: Please summarize the paper in your own words. This paper introduces the problem of determining the network activity profiling and its dual mode - land use classification, from a mobile traffic data. Authors propose to employ a standard exploratory factor analysis (EFA) to extract latent factor from the data and empirically shows that such activities can be identified by using those factors. It further claims that their results match with possibly a state of the art technique and provides additional insights in profiling. > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The paper studies an interesting data analysis problem that utilizes mobile network data and correlates it with other data sources for land usage classification. The paper is well written. > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The paper does not address the key issue - why EFA is suitable for network profiling, and, say, not PCA for that matter. Also, there is no reason to believe that linear relationships among variables in mobile traffic data hold. Since EFA presumes linearity in the data, it is not possible to discover any non-linear pattern from this method. > *** Detailed Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. The paper is well written and easy to follow. The authors apply a fairly standard EFA to extract latent factors and build a network profile (and similarly for land use classification) using those. Profiling results matches the known facts about human activities which were further compared against another results (which relies on hierarchical clustering). Authors also did a good job in interpreting the factors and its loading values. However, I didn't find any novel contribution by their method other than interpreting extracted factor components for providing few more insights in the data. Also, it would more compelling if results were compared against PCA or other dimension reduction technique to see how EFA dominates. Lastly, the paper should have addressed non-linearity in the data as it likely to exist in mobile traffic or at-least discuss the limitations of EFA in particular related to mobile traffic data. > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). Likely Reject (This paper should be rejected but I will not fight strongly against it) (2) Reviewer familiarity - Good knowledge (3) ======= Review 3 ======= > *** Paper Summary: Please summarize the paper in your own words. The paper develops a recipe for studying the behavior of bulky mobile traffic data using a technique called Exploratory Factor Analysis (EFA). Their method -- derived from a century ago paper on Psychology research-- is lucid, implementable and provides better (more detailed) results than the current state-of-the-art in spatio-temporal mobile data traffic analysis. > *** Strengths: What are the main reasons to accept the paper? You may comment on the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. The paper's most novel aspect is the use of well-known tools in psychology research to analyze mobile data traffic, which is shown to be a very suitable tool that provides richer data compared to state-of-the-art approaches. The problem addressed in the paper is very relevant, given the availability and accessibility of ubiquitous big data. The paper is well-written; the mathematical preliminaries are succinct yet clear. As far as my understanding goes, the authors are not developing any new technical module instead they are mapping their problem to the already known tool and they do it well. The main contribution of their work is in showing how EFA is instrumental in data analysis for both the cases: (1) Network activity profiling and (2) land use classification. > *** Weaknesses: What are the main reasons NOT to accept the paper? Again, think about the importance of the problems addressed, the novelty of the proposed solutions, the technical depth, and potential impact. Your overall rating should be supported by your review. - The paper does not develop new techniques, but applies known techniques from a different field. This can be viewed as a reason to reject the paper, though I would not. > *** Detailed Comments: Additional comments (if any) that you would like to provide to the authors. Please do not repeat what you stated above. If none, leave the following blank. The author aggregate the data from TIM dataset (in case of Network activity profiling) and from Orange dataset (for Land use classification) to get a median data (over hourly slots) and then use that for analysis. It would be good to see results on disaggregate data (not median). The authors say it is possible but it is unclear how. > *** Overall Rating: Your overall rating (based on strengths, weaknesses, and quality of writing). Likely Accept (This paper should be accepted but I will not champion it) (4) Reviewer familiarity - Good knowledge (3)