中国大学交通数据挖掘技术（Data Mining for Transportation）课后答案(mooc2023课后作业答案)
87 min read中国大学交通数据挖掘技术（Data Mining for Transportation）课后答案(mooc2023课后作业答案)
Test 1
1、中国Which one is 大学not the description of Data mining?
A、Extraction of interesting patterns or knowledge
B、交通技术Explorations and analysis 数据by automatic or semi-automatic means
C、Discover meaningful patterns from large quantities of data
D、挖掘Appropriate statistical analysis 课c课methods to analyze the data collected
2、Which one describes the right process of knowledge discovery?后答后作
A、Selection-Preprocessing-Transformation-Data mining-Interpretation/Evaluation
B、案m案Preprocessing-Transformation-Data mining- Selection- Interpretation/Evaluation
C、业答Data mining- Selection- Interpretation/Evaluation- Preprocessing-Transformation
D、中国Transformation-Data mining- election-Preprocessing- Interpretation/Evaluation
3、大学Which one is 交通技术not belong to the process of KDD?
A、Data mining
B、数据Data description
C、挖掘Data cleaning
D、课c课Data selection
4、Which one is not the right alternative name of data mining?
A、Knowledge extraction
B、Data archeology
C、Data dredging
D、Data harvesting
5、Which one is the nominal variables?
A、Occupation
B、Education
C、Age
D、Color
6、Which one is wrong about classification and regression?
A、Regression analysis is a statistical methodology that is most often used for numeric prediction.
B、We can construct classification models (functions) without some training examples.
C、Classification predicts categorical (discrete, unordered) labels.
D、Regression models predict continuous-valued functions.
7、Which one is wrong about clustering and outliers?
A、Clustering belongs to supervised learning.
B、Principles of clustering include maximizing intra-class similarity and minimizing interclass similarity.
C、Outlier analysis can be useful in fraud detection and rare events analysis.
D、Outlier means a data object that does not comply with the general behavior of the data.
8、About data process, which one is wrong?
A、Support vector machines do not belong to classification arithmetics.
B、When making data classification, we predict categorical labels excluding unordered one.
C、When making data characterization, we summarize the data of the class under study (the target class) in general terms.
D、When making data clustering, we would group data to form new categories.
9、Outlier arithmetics such as Isolation Forest can be used to detect traffic incident.
10、Support vector machines do not belong to classification arithmetics.
Week 2. Data pre-processing
Exercise 2
1、Which is not the reason we need to preprocess the data?
A、to save time
B、to make result meet our hypothesis
C、to avoid unreliable output
D、to eliminate noise
2、Which is not the major tasks in data preprocessing?
A、Clean
B、Integration
C、Transition
D、Reduction
3、Which is not the right way to fill in missing values？
A、Smart mean
B、Probable valve
C、Ignore
D、Falsify
4、Which one is not the right way to handle noise data?
A、Regression
B、Cluster
C、WT
D、Manual
5、How to construct new feature space by PCA?
A、New feature space by PCA is constructed by choosing the most important features you think.
B、New feature space by PCA is constructed by normalizing input data.
C、New feature space by PCA is constructed by selecting features randomly.
D、New feature space by PCA is constructed by eliminating the weak components to reduce the size of the data.
6、Which one is right about wavelet transforms?
A、Wavelet transforms store large fractions of the strongest of the wavelet coefficients.
B、Wavelet transforms are completely different from discrete Fourier transform (DFT).
C、Each transform has 2 functions: smoothing, difference.
D、Wavelet transforms means applying to pairs of data, resulting in two set of data of length L.
7、Which one is wrong about methods for discretization?
A、Histogram analysis and Binging are both unsupervised methods.
B、Clustering analysis only belongs to top-down split.
C、Interval merging by c2 Analysis can be applied recursively.
D、Decision-tree analysis is Entropy-based discretization.
8、Which one is wrong about Equal-width (distance) partitioning and Equal-depth (frequency) partitioning?
A、Equal-width (distance) partitioning is the most straightforward, but outliers may dominate presentation.
B、Equal-depth (frequency) partitioning Divides the range into N intervals, each containing approximately same number of samples.
C、The interval of the former one is not equal.
D、The number of tuples is the same when using the latter one.
9、Which one is not the right way to normalize data?
A、Min-max normalization
B、Simple scaling
C、Z-score normalization
D、Normalization by decimal scaling
10、Which one is the common used ways to sampling?
A、Simple random sample without replacement
B、Simple random sample with replacement
C、Stratified sample
D、All above
11、The purpose of correlation analysis is to identify redundant data.
12、Discretization means dividing the range of a continuous attribute into intervals.
Week 3. Instance based learning
Test 3
1、What’s the difference between eager learner and lazy learner?
A、Eager learners would generate a model for classification while lazy learner would not.
B、Eager learners classify the turple based on its similarity to the stored training turple while lazy learner not.
C、Eager learners simply store data (or does only a little minor processing) while lazy learner not.
D、Lazy learner would generate a model for classification while eager learner would not.
2、How to choose the optimal value for K?
A、Cross-validation can be used to determine a good value by using an independent dataset to validate the K values.
B、Low values for K (like k=1 or k=2) can be noisy and subject to the effect of outliers.
C、A large k value can reduce the overall noise so the value for 'k' can be as big as possible.
D、Historically, the optimal K for most datasets has been between 3-10.
3、What’s the major components in KNN?
A、How to measure similarity?
B、How to choose "k"?
C、How are class labels assigned?
D、How to decide the distance?
4、Which the following ways can be used to obtain attribute weight for Attribute-Weighted KNN?
A、Prior knowledge / experience.
B、PCA, FA (Factor analysis method).
C、Information gain.
D、Gradient descent, simplex methods and genetic algorithm.
5、At learning stage KNN would find the K closest neighbors and then decide classify K identified nearest label.
6、At classification stage KNN would store all instance or some typical of them.
7、Normalizing the data can solve the problem of different attributes have different value ranges.
8、By Euclidean distance or Manhattan distance, we can calculate the distance between two instances.
9、Data normalization before Measure Distance is generally to avoid errors caused by different dimensions, self-variations, or large numerical differences.
10、The way to obtain the regression for a new instance from the k nearest neighbors is to calculate the average value of k neighbors.
11、The way to obtain the classification for a new instance from the k nearest neighbors is to calculate the majority class of k neighbors.
12、The way to obtain instance weight for Distance-Weighted KNN is to calculate the reciprocal of the distance squared between object and neighbors.
Week 4. Decision Trees
Test 4
1、What types of nodes do a decision tree consist of?
A、Internal nodes test the value of particular features
B、Leaf nodes specify the class
C、Branch nodes decide the result
D、Root nodes decide the start point
2、How to compute information gain for continuous value attribute when using ID3?
A、Sort the value A in increasing order
B、Consider the midpoint between each pair of adjacent values as a possible split point
C、Select the minimum expected information requirement as the split-point
D、Spilt
3、Which is the typical algorithms to generate trees?
A、A horribly Intractable Algorithm
B、C4.5
C、CART
D、A Better Algorithm
4、Which one is right about underfitting and overfitting?
A、Underfitting means poor accuracy both for training data and unseen samples
B、Overfitting means high accuracy for training data but poor accuracy for unseen samples
C、Underfitting implies the model is too simple that we need to increase the model complexity
D、Overfitting occurs too many branches that we need to decrease the model complexity
5、Which one is right about pre-pruning and post-pruning?
A、Both of them are methods to deal with overfitting problem
B、Pre-pruning does not split a node if this would result in the goodness measure falling below a threshold
C、Post-pruning removes branches from a “fully grown” tree
D、It is easy to choose an appropriate threshold when making pre-pruning
6、What is the procedure of post-pruning?
A、First, consider the cost complexity of a tree
B、Then, for each internal node, N, compute the cost complexity of the subtree at N
C、And also compute the cost complexity of the subtree at N if it were to be pruned
D、At last, compare the two values. If pruning the subtree at node N would result in a smaller cost complexity, the subtree is pruned. Otherwise, the subtree is kept
7、The cost complexity pruning algorithm, namely pre-pruning, is used in CART
8、Gain ratio is used as attribute selection measure in C4.5 and the formula is GainRatio(A) = Gain(A)/ SplitInfo(A)
9、Rule is created for each part from its root to its leaf notes
10、ID3 use information gain as its attribute selection measure. And the attribute with the lowest information gain is chosen as the splitting attribute for note N
Exercise 4
1、Calculation of Information Gain of a Traffic Conflict Problem The students in Southeast University conducted a series of field observation of the occurrence of traffic conflict on December 14, 2019 at an intersection at Si-Pai-Lou Street and North Taiping Road in Nanjing, China. They also recorded the associated traffic volume, drivers’ demographical information, and visibility, as have been listed in Table 1. Table 1. The Rule Table for a Traffic Conflict Problem Traffic Volume Driver’ age License Less than 5 years Visibility Conflict high Young no fair no high Young no excellent no high Medium no fair yes high Medium yes fair yes medium Young no fair no medium Old yes fair yes medium Young yes excellent yes medium Medium no excellent yes medium Old no excellent no medium Old no fair yes low Old yes fair yes low Old yes excellent no low Medium yes excellent yes low Young yes fair yes (1) Plot the decision tree that can illustrate the traffic conflict problem in Table 1.（3 points） (2) Fill in Table 2 for the positive counts of conflict (pi) and negative counts of conflict (ni), and their totals.（3 points） Table 2. Positive and Negative Counts of Conflicts Traffic Volume pi ni Total Low Medium High Total (3) What is the probability of the class “conflict->yes” and the class “conflict -> no”?（2 points） (4) Calculate the information gained by branching on attribute “Traffic Volume” using ID3/C4.5 method.（4 points）
Week 5. Support Vector Machine
Test 5
1、What the feature of SVM?
A、Extremely slow, but are highly accurate
B、Much less prone to overfitting than other methods
C、Black box model
D、Provide a compact description of the learned model
2、Which is the typical common kernels?
A、Linear
B、Polynomial
C、Radial basis function (Gaussian kernel)
D、Sigmoid kernel
3、What adaptations can be made to allow SVM to deal with Multiclass Classification problem?
A、One versus rest (OVR)
B、One versus one (OVO）
C、Error correcting input codes (ECIC)
D、Error correcting output codes (ECOC).
4、What's the problem of OVR?
A、Sensitive to the accuracy of the confidence figures produced by the classifiers.
B、The scale of the confidence values may differ between the binary classifiers.
C、The binary classification learners see unbalanced distributions.
D、Only when the class distribution is balanced can balanced distributions attain.
5、Which one is right about the advantages of SVM?
A、They are accurate in high-dimensional spaces.
B、They are memory efficient.
C、The algorithm is not prone for over-fitting compared to other classification method.
D、The support vectors are the essential or critical training tuples.
6、Kernel trick was used to avoid costly computation and deal with mapping problems.
7、There is no structured way and no golden rules for setting the parameters in SVM.
8、Error correcting output codes (ECOC) is a kind of problem transformation techniques
9、Regression formulas including three types: linear, nonlinear and general form
10、If you have a big dataset, SVM is suitable for efficient computation
Exercise 5
1、In this exercise, you are required to use support vector machines (SVMs) to prediction incident duration. By this assignment on SVMs, you can get deep understanding of how to use SVMs. The data comes from the national incident management center for towing operations. These data were provided by towing officers, police and Rijkswaterstaat road-inspectors who perform incident handling. The data was collected from 1st May to 13th September 2005 on the region of Utrecht. You can find 1853 registrations of incident in total in the incidentduration.csv. Test_set.csv extract 50% of data and is used to test SVM, while train_set.csv contains the remaining and is used to train SVM. Attributes includes as follows: 1. Incident type (Stopped vehicle, lost load, accident); 2. Kind of vehicles involved (passenger cars, trucks, N/A); 3. Police required (yes, no) 4. Track research (yes, no) 5. Ambulance required (yes, no) 6. Fire brigades required (yes, no) 7. Repair service required (yes, no) 8. Tow truck required (yes, no) 9. Road inspector (yes, no) 10. Lane closer (yes, no) 11. Road repair required (yes, no) 12. Fluid to be cleaned (yes, no) 13. Damage of road equipment (yes, no) 14. Number of vehicles (Involved Single, two, more) 15. Type bergings task (onb, CMI, CMV) 16. By the week (workdays, weekend) 17. Start and end time (during peek hour, off peek hour) 18. Duration (short, long) Build prediction models with SVM to complete the following tasks. A. Use train_set.csv as train data set to build a model, and test on test_set.csv. Report the accuracy of train model and test model you get. (Suggestions: In data preprocess, you can deal with independent variables which are nominal variables by using one-hot representation, so that the corresponding value of each feature is guaranteed to be 0 or 1. It is easy to implement with get_dummies(data) function in the pandas package.) B. Use incidentduration.csv to build a model using 10-fold cross validation. Report the accuracy of train model and test model you get. C. Build a prediction model again after Feature Reduction (Keep 80% variance). Report the accuracy of train model and test model you get. D: Which model gives the highest accuracy on the test set? Why? Give you explanation.
Week 6. Outlier Mining
Test 6
1、Which description is right to describe outliers?
A、Outliers caused by measurement error
B、Outliers reflecting ground truth
C、Outliers caused by equipment failure
D、Outliers needed to be dropped out always
2、What is application case of outlier mining?
A、Traffic incident detection
B、Credit card fraud detection
C、Network intrusion detection
D、Medical analysis
3、Which one is the method to detect outliers?
A、Statistics-based approach
B、Distance-based approach
C、Bulk-based approach
D、Density-based approach
4、How to pick the right k by a heuristic method for density based outlier mining method?
A、K should be at least 10 to remove unwanted statistical fluctuations.
B、Pick 10 to 20 appears to work well in general.
C、Pick the upper bound value for k as the maximum of “close by” objects that can potentially be global outliers.
D、Pick the upper bound value for k as the maximum of “close by” objects that can potentially be local outliers.
5、Which one is right about three methods of outlier mining?
A、Statistics-based approach is simple and fast but difficult to deal with periodicity data and categorical data.
B、The efficiency of distance-based approach is low for the great data set in high dimensional space.
C、Distance-based approach cannot be used in multidimensional data set.
D、Density-based approach spends low cost on searching neighborhood.
6、Distance-based outlier Mining is not suitable to data set that does not fit any standard distribution model.
7、Statistic-based method needs to require knowing the distribution of the data and the distribution parameters in advance.
8、When identifying outliers with a discordancy test, the data point is considered as an outlier if it falls within the confidence interval.
9、Mahalanobis Distance accounts for the relative dispersions and inherent correlations among vector elements, which is different from Euclidean Distance.
10、An outlier is a data object that deviates significantly from the rest of the objects, as if it were generated by a different mechanism.
Exercise 6
1、Assignment 6 - Outlier mining You are required to use outlier mining methods to detect the outliers with given data sets. In a section of a city road, several cameras are set to collect the plate of vehicles from 2017-06-09 to 2017-06-12, as well as the date and time when passing the start point and the finish point. Travel time is calculated later. Time serial is another form of transformation from start time. So each instance contains 8 attributes, including serial number, license plate number, date and time passing start/end point, time serial and travel time. There are totally 4977 instances. You need to finish the following tasks. Task: (1) Use statistic-based approach to detect the outliers of travel time. Calculate the mean value and the variance of travel time. Write out the confidence interval. Take time serial as X-axis and the travel time as Y-axis. Plot the scatter diagram and mark the outliers you have recognized. (2) Use distance-based approach to detect the outliers of travel time. An object o in data set D is defined as an outlier with parameters r and π described as DB(r,π), if a fraction of the objects in D lie at a distance less than r from o is less than π, o is an outlier. Let parameter r vary from 0.1 to 0.3 with the step of 0.1, and π vary from 30 to 90 with the step of 30, find the outliers and the number of the outliers. You can use the Euclidian distance. (3) Use density-based approach to detect the outliers of travel time. With different k (from 3 to 400 with the step of 5), the number of neighbors, calculate the LOF for each data point. Set 2.0 as a threshold for LOF and an object is labeled as an outlier if its LOF exceeds 2.0. Firstly, take k value as X-axis and the number of outliers as Y-axis. Plot the line chart. Secondly, calculate the LOF for each data point and give the top 4 outliers. Use k=350 and the Euclidian distance.
Week 7. Ensemble Leaning
Test 7
1、How to deal with imbalanced data in 2-class classification?
A、Oversampling
B、Undersampling
C、Threshold-moving
D、Ensemble techniques
2、Which one is right when dealing with the class-imbalance problem?
A、Oversampling works by decreasing the number of minority positive tuples.
B、Undersampling works by increasing the number of majority negative tuples.
C、Smote algorithm adds synthetic tuples that are close to the minority tuples in tuple space.
D、Threshold-moving and ensemble methods were empirically observed to outperform oversampling and undersampling.
3、Which step is necessary when constructing an ensemble model?
A、Creating multiple data set
B、Constructing a set of classifiers from the training data
C、Combining predictions made by multiple classifiers to obtain final class label
D、Find the best performing predictions to obtain final class label
4、Ensembles tend to yield better results when there is a significant diversity among the base models.
5、Ensemble method cannot parallelizable because not every base classifier can be allocated to a different CPU.
6、To generate the single classifier, different model may be used to deal with different data subset.
7、In random forest, using a random selection of attributes at each node to determine the split.
8、Forest RI creates new attributes that are a linear combination of the existing attributes.
9、The principle threshold-moving is to move the decision threshold, so that the rare class tuples are easier to classify.
10、Neutral network classifiers can be used as classifiers in threshold-moving approach.
Exercise 7
1、Ensemble Learning[1] 1. Suppose you have trained three support vector machines, h1, h2, and h3, returning binary classifications (+1 or ?1). The observed accuracy of each of the hypotheses is 70%. Assuming that the errors of these hypotheses are independent, what is the predicted accuracy of an ensemble hypothesis H? 2. Suppose you have done one iteration of Adaboost to produce classifier h1 and find that h1 has the following results on the training data. (Assume the initial weights on the training data are uniform.) Example True Class Predicted Class (h1(x)) X1 +1 +1 X2 +1 +1 X3 -1 -1 X4 -1 +1 X5 -1 +1 (a) What is α1? (b) Assuming that the initial probability distribution w0 over the training data was uniform, give the probability distribution w1 that Adaboost would use in its next iteration (i.e., give the probability that would be assigned to each example). (c) Suppose you now run Adaboost for one additional iteration to produce classifier h2, and suppose that h2 has the following results on the training data: Example True Class Predicted Class (h2(x)) X1 +1 +1 X2 +1 -1 X3 -1 -1 X4 -1 -1 X5 -1 -1 What is α2? (d) At this point you decide to stop iterating Adaboost. You then run h1 and h2 on a new test instance x6; h1 classifies x6 as +1, and h2 classifies x6 as ?1. What is the resulting ensemble classifier H’s classification of x6? [1] These exercises come from ”EnsembleLearningExerciseSolution”(2019),[Online] Available: https://cat.pdx.edu/services/web/account-websites/,2020/5/28
Week 8 Clustering
Test 8
1、1. The clustering problem belongs to which subfield of machine learning
A、Supervised learning
B、Unsupervised learning
C、Semi-supervised learning
D、Deep learning
2、Which parameter is the hidden one in EM method for GMM estimations?
A、μ, Mean value for Gaussian distribution
B、, variations for Gaussian distribution
C、w, likelihood of one point belonging to one cluster
D、z, cluster membership
3、The general steps for clustering includes:
A、Feature selections
B、Proximity measurement
C、Clustering criteria and algorithms
D、Validation
4、Select the common methods for clustering.
A、K-Means
B、Classification and Regression Tree
C、DBSCAN
D、Gaussian Mixture Model
5、What points can be identified in the DBSCAN?
A、Outliers
B、Noise
C、Border
D、Core
6、In the K-means, the cluster mean points must be same as the real data points.
7、In the DBSCAN, the statement on density reachable is a symmetric one.
8、The GMM can yield the probability of one point belonging to one cluster.
Exercise 8
1、Summarize advantages and disadvantages of three common methods, including DBSCAN, GMM, and k-means, in case of big data and missing data. In 2D, assume 2 clusters, each randomly generate 10 points, then implement k-means. In 2D, randomly generate 100 points, and free to select the radius and MinPts (key parameters in DBSCAN), identify each point with noise, core, or border in python, then output the density-reachable points. To do GMM estimation, we commonly introduce EM method. please infer the expectation of hidden variable Z in e-step and log-likelihood in m-step.
Coursework
Analysis of Driving Behavior
1、In this coursework, you are required to use techniques of data mining to study the abnormal driving behavior. Please download the attachment and read the detail information of the coursework in coursework.docx file. You need to choose one to do from task 1 and task 2, and then choose one to do from task 3 and task 4. Hope you get good understanding after learning this course.
学习通交通数据挖掘技术
交通数据挖掘是一种通过对交通数据进行分析和建模，提取规律，发现知识，帮助交通管理者做出决策的技术。学习通交通数据挖掘技术是针对交通领域的数据挖掘技术的学习，下面为大家介绍学习通交通数据挖掘技术的相关知识。
1. 数据预处理
数据预处理是交通数据挖掘的第一步。由于交通数据的采集往往存在噪声和缺失值，因此需要对数据进行清洗，例如删除重复记录、填补缺失值等。此外，数据预处理还包括特征选择、特征缩放等操作，这些操作能够提高后续数据挖掘的效果。
2. 数据可视化
数据可视化是交通数据挖掘的重要环节之一，它可以帮助我们更好地理解数据。通过可视化技术，我们可以将数据以图表、图像等形式呈现出来，从而更好地发现数据中的规律和趋势。交通数据可视化的常见形式包括散点图、折线图、热力图等。
3. 数据挖掘算法
数据挖掘算法是交通数据挖掘的核心。常见的交通数据挖掘算法包括聚类、关联规则挖掘、分类、预测等。其中，聚类是将数据集分成不同的组的过程，关联规则挖掘是发现数据集中项之间的关联关系，分类是根据数据的属性将其分成不同类别，预测则是通过已有数据预测未来的数据。
4. 实战案例
以下是一些交通数据挖掘实战案例：
4.1 道路拥堵预测
通过对交通流量数据进行分析，可以预测道路拥堵的情况。这可以帮助交通管理者及时采取措施，避免道路拥堵造成的交通事故和交通堵塞。
4.2 公交线路优化
通过对公交车路线、车速、乘客量等数据进行分析，可以优化公交线路，提高公交系统的效率和服务质量。
4.3 交通事故预测
通过对历史交通事故数据进行分析，可以预测未来的交通事故发生情况，并采取相应的措施减少交通事故的发生。
5. 总结
学习通交通数据挖掘技术是一项重要的技术，它可以帮助交通管理者更好地了解交通数据，发现数据中的规律和趋势，并采取相应的措施，优化交通系统。
中国大学交通数据挖掘技术
随着城市化进程的加快，人口迁移、城市交通拥堵等问题越来越突出，如何科学合理地规划城市交通成为了一个亟待解决的问题。而交通数据挖掘技术提供了一个有效的解决方案。
什么是交通数据挖掘技术
数据挖掘是一种从大量数据中寻找有用信息的过程。交通数据挖掘技术是将数据挖掘应用于交通领域，通过分析交通相关的大量数据，找到隐藏在数据中的规律和模式，从而帮助城市交通规划者制定更好的交通规划，提高城市交通运输效率。
交通数据挖掘技术的应用
交通流预测
通过对历史交通数据的分析，可以预测未来的交通流量，从而提前做好交通调度和规划工作，减轻交通拥堵的程度。
路网优化
通过对路网数据的分析，可以发现交通拥堵的瓶颈和热点，从而优化路网、改善交通状况。
交通事故预测
通过对历史交通事故数据的分析，可以预测未来的交通事故发生概率，从而采取相应的措施，防止和减少交通事故的发生。
交通调度
通过对交通数据的实时监测和分析，可以动态调整交通信号、调度公交车、出租车等交通工具，从而提高交通效率。
交通数据挖掘技术的挑战
交通数据挖掘技术的应用面广泛，但也面临着一些挑战。
数据采集难度大
交通数据源分散，数据来源多样，数据采集难度大，包括交通视频监控、停车场记录、道路传感器等多个方面，数据缺失和数据质量的问题也比较严重。
数据处理难度大
交通数据量大，数据处理难度大，需要建立数据模型和算法，对数据进行分析和挖掘。同时，数据处理需要保证数据的准确性和实时性，这也对算法和模型提出了更高的要求。
数据隐私保护问题
交通数据中包含大量个人隐私信息，如车牌号、车主信息等，如何在数据挖掘的过程中保护个人隐私成为了一个重要的问题。
结论
交通数据挖掘技术为城市交通规划和交通管理提供了有效手段，但也需要面对一些挑战。未来，随着技术的不断发展和完善，交通数据挖掘技术将在城市交通领域发挥越来越重要的作用。
