Pith. sign in

REVIEW 5 major objections 5 minor 78 references

A systematic data characteristic understanding framework towards physical-sensor big data challenges

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a systematic framework that profiles physical-sensor data across six Vs with 30 statistical indicators and links the resulting profile to preprocessing recommendations.

desk verdict Useful 6Vs checklist with real time-series indicators, but the headline scores don't reproduce from the equations as written. read the letter →

arxiv 2501.12720 v1 pith:CTCAAO7G submitted 2025-01-22 cs.IR

Zhipeng Ma , Bo N{o}rregaard J{o}rgensen , Zheng Grace Ma This is my paper · ORCID
classification cs.IR
keywords Bigdatacharacteristics6VsmodelPhysical-sensorpreprocessingminingchallengesTimeseriesStatisticalindicators
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that physical-sensor data can be understood before any analytics by computing a profile of 30 statistical indicators organized under the six Vs of big data: volume, variety, velocity, veracity, value, and variability. The aim is to turn data-quality intuition into numbers that reveal which big-data challenges a dataset will pose, so that preprocessing can be chosen deliberately instead of by default. The framework adds time-related indicators (timestamps, intervals, duplicates, missing spans) that the authors argue earlier Vs models lack, and it links every dimension to a specific preprocessing recommendation. Two case studies on furnace sensor data and bus telemetry data illustrate the pipeline and produce per-dimension scores for each dataset. If the approach works, data scientists gain a standardized, data-driven way to compare datasets and plan data cleaning.

What carries the argument

The carrying mechanism is the 6Vs model together with the evaluation metrics and the three-stage pipeline that computes them. Each dimension has a formula: Vol(D)=NF(D)×NI(D); Varie(D) is PSD/PSSD (or +∞ for fully structured data); Vel(D) is the speed of data producing; Ver(D) is a weighted sum of four data-quality percentages; Val(D) is the fraction of invalid statistical indicators among the 15 distribution factors; and Varia(D) is a weighted sum of normalized standard deviation, outlier rate, and correlation validity. The pipeline implements the framework in three stages, namely timestamp understanding, value understanding, and feature understanding, which detect duplicate timestamps and irregular intervals, compute data-format and missing-value statistics, then recompute timestamps and cross-correlations. The formulas and thresholds (e.g., IQR-based outlier bounds, a 0.7 correlation cutoff, equal default weights) are what make the framework quantitative rather than qualitative.

What would settle it

Compute the Value metric on the two case-study datasets after adopting a specific rule for judging an indicator invalid; if different reasonable rules change the Value scores materially, or the paper's numbers cannot be reproduced from its own tables, the Value dimension is not a well-defined quantification.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a 6Vs model populated with 30 statistical indicators can unveil the data characteristics of physical-sensor data and thereby identify potential challenges in subsequent analytics. Each V is assigned a set of computable indicators and an evaluation metric: volume is the product of feature and instance counts, variety is the ratio of structured to unstructured data, velocity is the sensor update speed, veracity is a weighted average of format errors, abnormal spikes, irregular time intervals, duplicate timestamps, and missing values, value is the percentage of invalid statistical indicators, and variability is a weighted blend of normalized standard deviation, outlier rate, and weak cross-correlation. The paper shows two case studies in which these scores and indicators point to concrete preprocessing decisions, such as increasing the sampling rate, removing segments with long missing spans, and dropping features that are constantly zero. The authors also argue, from their indicator set, that the framework generalizes beyond sensor data to other tabular and time-series datasets.

Load-bearing premise

The load-bearing premise is that the framework's evaluation metrics quantify the six Vs objectively, and in particular the Value metric, defined as the percentage of invalid indicators, never specifies what makes an indicator invalid, so the reported Value scores cannot be reproduced from the paper.

Editorial extensions

If this is right

  • Data scientists can compute a six-number quality profile for any physical-sensor dataset before cleaning, making preprocessing choices explicit and comparable across projects.
  • Time-related problems, such as irregular intervals, duplicate timestamps, and long missing spans, become visible at the profiling stage instead of surfacing as modeling errors.
  • Because the indicators are data-driven and dataset-independent, the framework can serve as a common yardstick for datasets from different sectors and sensor systems.
  • The paper's case-study scores provide concrete reference points for what furnace and bus telemetry datasets look like under the 6Vs profile.
  • With weights and thresholds treated as tunable parameters, the framework can be adapted to application-specific quality requirements rather than relying on fixed defaults.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Value metric depends on a rule for classifying an indicator as invalid, a rule the paper never states; until it is specified, the reported Value scores are not reproducible from the published text.
  • A natural testable extension is to make the framework predictive: learn a mapping from the 30-indicator vector to preprocessing outcomes, allowing the profile itself to recommend actions rather than leaving the link to human reading.
  • The equal-weight default in the veracity and variability metrics is a strong modeling choice; under different weights the ordering of which dataset looks worse could change, so cross-dataset comparisons should report the chosen weights.
  • The same indicator set could be applied to non-sensor tabular and financial time series, which the paper hints at; a quick validation would be to profile such datasets and check whether the resulting challenges align with known data-quality issues.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a six-V (Volume, Variety, Velocity, Veracity, Value, Variability) framework for quantitatively characterizing physical-sensor data. Each V dimension is populated with statistical indicators and an evaluation metric, and the framework links each dimension to specific big-data challenges plus preprocessing recommendations. A three-stage pipeline (timestamp understanding, value understanding, feature understanding) implements the framework, and two case studies--foundry furnace data and public-transport bus data--demonstrate it. The claimed contribution is an objective, data-driven profiling method that reveals potential analytics challenges before preprocessing is applied.

Significance. The framework addresses a real gap, because most existing Vs models are qualitative and physical-sensor time series require timestamp-related indicators. The 30-indicator catalog, including time-interval, duplicate-timestamp, seasonality, and autocorrelation measures, is a concrete and potentially useful contribution. The two case studies are realistic, and the pipeline is described in enough detail to be implemented. However, the quantitative claims are currently undermined by several definitional and arithmetic inconsistencies in the evaluation metrics, so the framework cannot yet be used as a reproducible profiling tool on the basis of this manuscript. No code or data is released, so the numerical results are not independently verifiable from the text alone.

major comments (5)
  1. [Section 3.2.5, Eq. (11)] The Value metric is defined as the percentage of invalid indicators, but neither the term 'invalid indicator' nor the way any indicator is judged invalid is defined anywhere in the manuscript. As a result, the reported Value scores 0.1571 (Table 8) and 0.36 (Table 11) cannot be recomputed from the indicator values in Tables 10 and 13. Without a counting rule or a validity threshold, the Value dimension does not provide a reproducible quantitative measurement, which is the central promise of the framework.
  2. [Section 3.2.2, Eq. (5)] The prose states that the Variety metric is the ratio between structured data and non-structured data and that fully structured datasets receive '+∞', but the displayed formula is Vari(D) = PSD(D)/(PSD(D)+PSSD(D)). For an all-structured dataset this gives 1 (or a division-by-zero if PSSD=0 is treated as a zero denominator), not '+∞', and it omits PUD entirely. The formula should be reconciled with the prose, for example as PSD/(PUD+PSSD), and the '+∞' entries in Tables 8 and 11 should be updated accordingly.
  3. [Section 3.2.4, Eq. (10)] The Veracity metric is not fully specified. The term NAS(D)/(NI(D)-2) uses an unexplained offset of 2, with no definition of which instances are excluded, and the surrounding text says 'the values of all five weights are set at 1/4' even though only four weights W41-W44 are defined. Because the metric is a weighted sum, these ambiguities change the reported Veracity scores (1.128e-4 and 3.818e-3 in Tables 8 and 11) and must be clarified before the scores are reproducible.
  4. [Section 3.2.6, Eq. (16) and Figures 8/10] Equation 16 is described as the cross correlation with time delay k, but the displayed formula is the ordinary sample correlation coefficient with no lag variable k and no alignment of the two series; the text and figures instead use a 300-second delay. As written, the equation does not support the cross-correlation indicator on which the variability analysis and the feature-relationship conclusions in Section 6 rely. Please provide the lag-dependent definition actually used, including boundary handling.
  5. [Section 5.2.1, Tables 8-9] The headline Volume score in Table 8 is 11,317,010, but Eq. (1) with the indicators of Table 9 gives 7 x 1,624,430 = 11,371,010. This discrepancy of 54,000 in the first reported evaluation score indicates that the numerical results have not been proofread, and the entire set of reported scores should be rechecked for consistency.
minor comments (5)
  1. [Sections 5.2.1 and 5.2.2] The text says 'The data validity dimension involves standard deviation...' in both case-study sections; from the table contents this should read 'variability dimension'.
  2. [Sections 6.1.1 and 6.1.2] The headings and opening sentences refer to 'challenges in the data variety' when discussing data formats, missing values, duplicates, and abnormal spikes; those are veracity challenges, not variety challenges.
  3. [Section 5.2.1, Table 10] The text 'statesmodels 0.13.5' should be 'statsmodels 0.13.5'.
  4. [Declarations] 'No appliable' appears twice; it should be 'Not applicable'.
  5. [Tables 4 and 5] The header rows of the two factor tables are incomplete; the cells for the listed factor names are empty, which makes the structure of the tables difficult to parse.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation metrics are explicit composites of the indicators and no fitted parameter or external prediction reduces to its own inputs.

full rationale

The paper's evaluation metrics (Eq. 10 and Eq. 17) are explicitly defined as weighted combinations of the very indicators they summarize, and Section 6's challenge analysis restates those indicator values; this is a descriptive aggregation and interpretation, not a derivation of an independent result from data. No parameter is fitted to a subset and then 'predicted' on a related quantity, and no claim is made that the framework predicts an external outcome from the indicators. The self-citations [76,77] are peripheral pointers to feature-engineering and causal-inference techniques and are not load-bearing for the framework's central argument. The paper does contain internal correctness problems that are not circularity: Eq. 5's formula does not match its stated 'ratio between structured and non-structured data' and yields 1 rather than '+infinity' in the all-structured case, and Eq. 11's 'invalid indicators' criterion is never defined, making the reported Value scores non-reproducible. However, under the required standard that circularity must be evidenced by a specific reduction to the paper's own inputs, no such reduction is present here.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The framework relies on hand-chosen weights, thresholds, and definitions, plus assumptions that the 6Vs dimensions are non-overlapping and that sensor data are tabular with timestamps. No invented entities are introduced.

free parameters (6)
  • Veracity metric weights W41-W44 = 1/4 each by default
    Hand-chosen equal weights in Eq. 10; adjustable per application, no guidance or sensitivity analysis.
  • Variability metric weights W51-W53 = 1/3 each by default
    Hand-chosen equal weights in Eq. 17; no justification.
  • Outlier detection IQR multiplier = 1.5
    Standard Tukey boxplot rule, but adjustable; no justification for sensor data.
  • High correlation threshold = 0.7
    Threshold for VC in Eq. 20, chosen without domain justification.
  • Missing span thresholds = furnace: 30 min / 6 h; bus: 1 min / 30 min
    Short/medium/long missing span definitions differ across case studies, chosen per application.
  • Seasonality detection period = unspecified
    The seasonal_decompose model requires a period; the paper does not state the period used to declare 'No seasonality'.
assumptions (6)
  • domain assumption 6Vs model dimensions have mutual and non-overlapping definitions
    Section 2.2 and Section 3.1 assert the six Vs have clear boundaries based on literature [13,23,46]; this is assumed for the framework's structure.
  • domain assumption Physical-sensor data are time-related tabular data with timestamps
    Section 3.1 defines the scope; framework may not apply to non-tabular or non-timestamped sensor streams.
  • domain assumption Missing physical-sensor data are mostly missing at random (MAR)
    Section 3.2.4 assumes MAR based on refs [62,63]; if data are MNAR, the proposed handling logic would need revision.
  • domain assumption Standard statistical indicators (skewness, kurtosis, autocorrelation, cross-correlation, seasonal decomposition) adequately capture data value and variability
    Tables 3-5 and Section 3.2.5/3.2.6 select these features without a formal criterion.
  • standard math statsmodels seasonal_decompose model correctly identifies seasonality
    Section 5.2.1 applies seasonal_decompose; the paper does not verify the decomposition or its period parameter.
  • ad hoc to paper Evaluation metrics are meaningful summaries of data quality
    Eqs. 10 and 17 are constructed ad hoc as weighted sums of indicators with arbitrary weights; their validity as quality scores is assumed, not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A systematic data characteristic understanding framework towards physical-sensor big data challenges." pith.science (2026). https://pith.science/paper/CTCAAO7G

@misc{pith2026250112720,
  author       = {Pith},
  title        = {Pith review of: A systematic data characteristic understanding framework towards physical-sensor big data challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CTCAAO7G}},
  note         = {Machine review of arXiv:2501.12720}
}
read the original abstract

Big data present new opportunities for modern society while posing challenges for data scientists. Recent advancements in sensor networks and the widespread adoption of IoT have led to the collection of physical-sensor data on an enormous scale. However, significant challenges arise in high-quality big data analytics. To uncover big data challenges and enhance data quality, it is essential to quantitatively unveil data characteristics. Furthermore, the existing studies lack analysis of the specific time-related characteristics. Enhancing the efficiency and precision of data analytics through the big data lifecycle requires a comprehensive understanding of data characteristics to address the hidden big data challenges. To fill in the research gap, this paper proposes a systematic data characteristic framework based on a 6Vs model. The framework aims to unveil the data characteristics in terms of data volume, variety, velocity, veracity, value, and variability through a set of statistical indicators. This model improves the objectivity of data characteristic understanding by relying solely on data-driven indicators. The indicators related to time-related characteristics in physical-sensor data are also included. Furthermore, the big data challenges are linked to each dimension of the 6Vs model to gain a quantitative understanding of the data challenges. Finally, a pipeline is developed to implement the proposed framework, and two case studies are conducted to illustrate the process of understanding the physical-sensor data characteristics and making recommendations for data preprocessing to address the big data challenges. The proposed framework is able to analyze the characteristics of all physical-sensor data, therefore, identifying potential challenges in subsequent analytics, and providing recommendations for data preprocessing.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 55 canonical work pages

  1. [1]

    Challenges of big data analysis

    Fan J, Han F, Liu H. Challenges of big data analysis. Natl Sci Rev. 2014;1(2):293- 314. https://doi.org/10.1093/nsr/nwt032

    doi:10.1093/nsr/nwt032 2014

  2. [2]

    Fair data enabling new horizons for materials research

    Scheffler M, Aeschlimann M, Albrecht M, Bereau T, Bungartz H -J, Felser C, et al. Fair data enabling new horizons for materials research. Nature. 2022;604(7907):635- 42. https://doi.org/10.1038/s41586-022- 04501-x

    doi:10.1038/s41586-022- 2022

  3. [3]

    Uncertainty in big data analytics: Survey, opportunities, and challenges

    Hariri RH, Fredericks EM, Bowers KM. Uncertainty in big data analytics: Survey, opportunities, and challenges. J Big Data. 2019;6:44. https://doi.org/10.1186/s40537-019-0206-3

    work page doi:10.1186/s40537-019-0206-3 2019

  4. [4]

    https://www.statista.com/statistics/871513/worldwide-data-created/

    Volume of data/information created, captured, copied, and consumed worldwide from 2010 to 2020, with forecasts from 2021 to 2025. https://www.statista.com/statistics/871513/worldwide-data-created/. Accessed 16 November 2023

    work page 2010

  5. [5]

    Big data for healthcare industry 4.0: Applications, challenges and future perspectives

    Karatas M, Eriskin L, Deveci M, Pamucar D, Garg H. Big data for healthcare industry 4.0: Applications, challenges and future perspectives. Expert Syst Appl. 2022;200:116912. https://doi.org/10.1016/j.eswa.2022.116912. 53

    work page arXiv 2022

  6. [6]

    Beyond the hype: Big data concepts, methods, and analytics

    Gandomi A, Haider M. Beyond the hype: Big data concepts, methods, and analytics. Int J Inf Manage. 2015;35(2):137-44. https://doi.org/10.1016/j.ijinfomgt.2014.10.007

    work page doi:10.1016/j.ijinfomgt.2014.10.007 2015

  7. [7]

    Big data quality framework: A holistic approach to continuous quality management

    Taleb I, Serhani MA, Bouhaddioui C, Dssouli R. Big data quality framework: A holistic approach to continuous quality management. J Big Data. 2021;8:76. https://doi.org/10.1186/s40537-021-00468-0

    work page doi:10.1186/s40537-021-00468-0 2021

  8. [8]

    A novel rigorous measurement model for big data quality characteristics

    Zou H, Xiang K, editors. A novel rigorous measurement model for big data quality characteristics. 2022 IEEE International Conference on Big Data (Big Data); 2022: IEEE. https://doi.org/10.1109/BigData55660.2022.10020564

    arXiv 2022

Show all 78 references
  1. [9]

    Trends and future perspective challenges in big data

    Naeem M, Jamal T, Diaz-Martinez J, Butt SA, Montesano N, Tariq MI, et al. Trends and future perspective challenges in big data. Advances in Intelligent Data Analysis and Applications: Proceeding of the Sixth Euro-China Conference on Intelligent Data Analysis and Applications, ...

    2019 doi

  2. [10]

    big data

    Hashem IAT , Yaqoob I, Anuar NB, Mokhtar S, Gani A, Khan SU. The rise of “big data” on cloud computing: Review and open research issues. Inf Syst. 2015;47:98-115. https://doi.org/10.1016/j.is.2014.07.006

    2015 doi

  3. [11]

    3D data management: Controlling data volume, velocity and variety

    Laney D. 3D data management: Controlling data volume, velocity and variety. META group research note. 2001;6(70):1

    2001

  4. [12]

    A zero emission neighbourhoods data management architecture for smart city scenarios: Discussions toward 6vs challenges

    Sinaeepourfard A, Krogstie J, Petersen SA, Gustavsen A, editors. A zero emission neighbourhoods data management architecture for smart city scenarios: Discussions toward 6vs challenges. 2018 International Conference on Information and Communication Tec hnology Convergence (ICT...

    2018

  5. [13]

    A model for unpacking big data analytics in high -frequency trading

    Seddon JJ, Currie WL. A model for unpacking big data analytics in high -frequency trading. J Bus Res. 2017;70:300-7. https://doi.org/10.1016/j.jbusres.2016.08.003

    2017 doi

  6. [14]

    A comprehensive scenario agnostic data lifecycle model for an efficient data complexity management

    Sinaeepourfard A, Garcia J, Masip -Bruin X, Marín -Tordera E. A comprehensive scenario agnostic data lifecycle model for an efficient data complexity management. 2016 IEEE 12th International Conference on e-Science (e-Science); 2016 23-27 Oct. 2016. https://doi.org/10.1109/eScie...

    2016

  7. [15]

    Toward a novel measurement framework for big data (mega)

    Bhardwaj D, Ormandjieva O. Toward a novel measurement framework for big data (mega). 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC); 2021: IEEE. https://doi.org/10.1109/COMPSAC51774.2021.00235

    2021

  8. [16]

    Seven v's of big data understanding big data to extract value

    Uddin MF, Gupta N. Seven v's of big data understanding big data to extract value. Proceedings of the 2014 Zone 1 Conference of the American Society for Engineering Education; 2014: IEEE. https://doi.org/10.1109/ASEEZone1.2014.6820689

    2014

  9. [17]

    A study of big data analytics using apache spark with python and scala

    Gupta YK, Kumari S. A study of big data analytics using apache spark with python and scala. 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS); 2020: IEEE. https://doi.org/10.1109/ICISS49785.2020.9315863

    2020

  10. [18]

    Recent quality models in bigdata applications

    Hattawi W, Shaban S, Al Shawabkah A, Alzu’bi S. Recent quality models in bigdata applications. 2021 International Conference on Information Technology (ICIT); 2021: IEEE. https://doi.org/10.1109/ICIT52682.2021.9491629

    2021

  11. [19]

    Defining big data

    Emmanuel I, Stanier C. Defining big data. Proceedings of the International Conference on Big Data and Advanced Wireless Technologies; 2016. https://doi.org/10.1145/3010089.3010090

    2016

  12. [20]

    Towards a comprehensive data lifecycle model for big data environments

    Sinaeepourfard A, Garcia J, Masip-Bruin X, Marín-Torder E. Towards a comprehensive data lifecycle model for big data environments. Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies; 2016. https://doi.org/10.1145/30062...

    2016

  13. [21]

    Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics

    Ghasemaghaei M. Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics. Int J Inform Manage. 2021;57:102055. https://doi.org/10.1016/j.ijinfomgt.2019.102055

    2021

  14. [22]

    A global manufacturing big data ecosystem for fault detection in predictive maintenance

    Yu W, Dillon T, Mostafa F, Rahayu W, Liu Y . A global manufacturing big data ecosystem for fault detection in predictive maintenance. IEEE Trans Ind Inform. 2019;16(1):183 -92. https://doi.org/10.1109/TII.2019.2915846

    2019

  15. [23]

    Big data perspective for driver/driving behavior

    Terzi R, Sagiroglu S, Demirezen MU. Big data perspective for driver/driving behavior. IEEE Intell Transp Syst Mag. 2018;12(2):20-35. https://doi.org/10.1109/MITS.2018.2879220. 55

    2018

  16. [24]

    A trustworthy privacy preserving framework for machine learning in industrial IoT systems

    Arachchige PCM, Bertok P , Khalil I, Liu D, Camtepe S, Atiquzzaman M. A trustworthy privacy preserving framework for machine learning in industrial IoT systems. IEEE Trans Ind Inform. 2020;16(9):6092 -102. https://doi.org/10.1109/TII.2020.2974555

    2020

  17. [25]

    Mu l ti-sensor information fusion based on machine learning for real applications in human activity recognition: State -of-the-art and research challenges

    Q i u S , Z h a o H , J i a n g N , W a n g Z , Li u L, A n Y , e t a l . Mu l ti-sensor information fusion based on machine learning for real applications in human activity recognition: State -of-the-art and research challenges. Information Fusion. 2022;80:241-65

    2022

  18. [26]

    Internet of things in industries: A survey

    Da Xu L, He W , Li S. Internet of things in industries: A survey. IEEE Trans ind inform. 2014;10(4):2233-43. https://doi.org/10.1109/TII.2014.2300753

    2014

  19. [27]

    Industry 4.0: A survey on technologies, applications and open research issues

    Lu Y . Industry 4.0: A survey on technologies, applications and open research issues. J Ind Inf Integr. 2017;6:1-10. https://doi.org/10.1016/j.jii.2017.04.005

    2017 doi

  20. [28]

    Privacy-aware traffic flow prediction based on multi-party sensor data with zero trust in smart city

    Wang F, Li G, Wang Y , Rafique W , Khosravi MR, Liu G, et al. Privacy-aware traffic flow prediction based on multi-party sensor data with zero trust in smart city. ACM Trans Internet Technol. 2023;23(3):1 -19. https://doi.org/10.1145/3511904

    2023 doi

  21. [29]

    Serverless data pipeline approaches for IoT data in fog and cloud computing

    Poojara SR, Dehury CK, Jakovits P , Srirama SN. Serverless data pipeline approaches for IoT data in fog and cloud computing. Futur Gener Comp Syst. 2022;130:91 -105. https://doi.org/10.1016/j.future.2021.12.012

    2022 doi

  22. [30]

    Usman M, Jahanger A, Makhdum MSA, Balsalobre -Lorente D, Bashir A. How do financial development, energy consumption, natural resources, and globalization affect arctic countries' economic growth and environmental quality? An advanced panel data simulati on. Energy. 2022;241:1225...

    2022

  23. [31]

    Human activity recognition in artificial intelligence framework: A narrative review

    Gupta N, Gupta SK, Pathak RK, Jain V, Rashidi P , Suri JS. Human activity recognition in artificial intelligence framework: A narrative review. Artif Intell Rev. 2022;55(6):4755-808. https://doi.org/10.1007/s10462- 021-10116-x

    2022 doi

  24. [32]

    Dalif: A data lifecycle framework for data-driven governments

    Shah SIH, Peristeras V, Magnisalis I. Dalif: A data lifecycle framework for data-driven governments. J Big Data. 2021;8:89. https://doi.org/10.1186/s40537-021-00481-3. 56

    2021 doi

  25. [33]

    Big-data approaches lead to an increased understanding of the ecology of animal movement

    Nathan R, Monk CT, Arlinghaus R, Adam T, Alós J, Assaf M, et al. Big-data approaches lead to an increased understanding of the ecology of animal movement. Science. 2022;375(6582):eabg1780. https://doi.org/10.1126/science.abg1780

    2022 doi

  26. [34]

    Big data for creating and capturing value in the digitalized environment: Unpacking the effects of volume, variety, and veracity on firm performance

    Cappa F, Oriani R, Peruffo E, McCarthy I. Big data for creating and capturing value in the digitalized environment: Unpacking the effects of volume, variety, and veracity on firm performance. J Prod Innov Manage. 2021;38(1):49-67. https://doi.org/10.1111/jpim.12545

    2021 doi

  27. [35]

    A model of the data (life) cycles with application to quality

    Levitin AV, Redman TC. A model of the data (life) cycles with application to quality. Inf Softw Technol. 1993;35(4):217-23. https://doi.org/10.1016/0950-5849(93)90069-F

    1993 doi

  28. [36]

    Reference architecture and classification of technologies, products and services for big data systems

    Pääkkönen P , Pakkala D. Reference architecture and classification of technologies, products and services for big data systems. Big Data Res. 2015;2(4):166-86. https://doi.org/10.1016/j.bdr.2015.01.001

    2015 doi

  29. [37]

    Toward data mining engineering: A software engineering approach

    Marbán O, Segovia J, Menasalvas E, Fernández -Baizán C. Toward data mining engineering: A software engineering approach. Inform Syst. 2009;34(1):87-107. https://doi.org/10.1016/j.is.2008.04.003

    2009 doi

  30. [38]

    CRISP-DM twenty years later: From data mining processes to data science trajectories

    Martínez-Plumed F, Contreras-Ochando L, Ferri C, Hernández-Orallo J, Kull M, Lachiche N, et al. CRISP-DM twenty years later: From data mining processes to data science trajectories. I EEE T Knowl Data En. 2021;33(8):3048-61. https://doi.org/10.1109/Tkde.2019.2962680

    2021

  31. [39]

    C RISP-DM: Towards a standard process model for data mining

    Wirth R, Hipp J. C RISP-DM: Towards a standard process model for data mining. Proceedings of the 4th international conference on the practical applications of knowledge discovery and data mining; 2000: Manchester

    2000

  32. [40]

    APREP-DM: A framework for automating the pre-processing of a sensor data analysis based on CRISP-DM

    Nagashima H, Kato Y . APREP-DM: A framework for automating the pre-processing of a sensor data analysis based on CRISP-DM. 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops); 2019: IEEE. https://doi.org/10.1109/PERCOMW.201...

    2019

  33. [41]

    Variations of length of stay: A case study using control charts in the CRISP-DM framework

    Azadeh-Fard N, Megahed FM, Pakdil F. Variations of length of stay: A case study using control charts in the CRISP-DM framework. Int J Six Sigma Competitive Advant. 2019;11(2 -3):204-25. https://doi.org/10.1504/IJSSCA.2019.101418. 57

    2019

  34. [42]

    C RISP data mining methodology extension for medical domain

    Niaksu O. C RISP data mining methodology extension for medical domain. Balt J Mod Comput. 2015;3(2):92

    2015

  35. [43]

    Evaluating frameworks for implementing machine learning in signal processing: A comparative study of CRISP-DM, SEMMA and KDD

    Dåderman A, Rosander S. Evaluating frameworks for implementing machine learning in signal processing: A comparative study of CRISP-DM, SEMMA and KDD. 2018. https://www.diva - portal.org/smash/get/diva2:1250897/FULLTEXT01.pdf. Accessed 26 June 2022

    2018

  36. [44]

    Synthesizing CRISP-DM and quality management: A data mining approach for production processes

    Schäfer F, Zeiselmair C, Becker J, Otten H. Synthesizing CRISP-DM and quality management: A data mining approach for production processes. 2018 IEEE International Conference on Technology Management, Operations and Decisions (ICTMOD); 2018: IEEE. https://doi.org/10.1109/ITMC.2...

    2018

  37. [45]

    On applicability of big data analytics in the closed -loop product lifecycle: Integration of crisp-dm standard

    Gholamzadeh Nabati E, Thoben K -D. On applicability of big data analytics in the closed -loop product lifecycle: Integration of crisp-dm standard. Product Lifecycle Management for Digital Transformation of Industries; 2016: Springer. https://doi.org/10.1007/978-3-319-54660-5_41

    2016 doi

  38. [47]

    Addressing big data issues in scientific data infrastructure

    Demchenko Y , Grosso P , De Laat C, Membrey P . Addressing big data issues in scientific data infrastructure. 2013 International conference on collaboration technologies and systems (CTS); 2013: IEEE. https://doi.org/10.1109/CTS.2013.6567203

    2013

  39. [48]

    A comprehensive survey on feature selection in the various fields of machine learning

    Dhal P , Azad C. A comprehensive survey on feature selection in the various fields of machine learning. Appl Intell. 2022;52:4543-81. https://doi.org/10.1007/s10489-021-02550-9

    2022 doi

  40. [49]

    Feature dimensionality reduction: A review

    Jia W, Sun M, Lian J, Hou S. Feature dimensionality reduction: A review. Complex Intell Syst. 2022;8:2663-

    2022

  41. [50]

    Use of large-scale hrqol datasets to generate individualised predictions and inform patients about the likely benefit of surgery

    Gutacker N, Street A. Use of large-scale hrqol datasets to generate individualised predictions and inform patients about the likely benefit of surgery. Qual Life Res. 2017;26(9):2497 -505. https://doi.org/10.1007/s11136-017-1599-0

    2017 doi

  42. [51]

    A review of industrial big data for decision making in intelligent manufacturing

    Li C, Chen Y , Shang Y . A review of industrial big data for decision making in intelligent manufacturing. Eng Sci Technol. 2022;29:101021. https://doi.org/10.1016/j.jestch.2021.06.001. 58

    2022 doi

  43. [52]

    Combining structured and unstructured data for predictive models: A deep learning approach

    Zhang D, Yin C, Zeng J, Yuan X, Zhang P . Combining structured and unstructured data for predictive models: A deep learning approach. BMC Med Inform Decis Mak. 2020;20:280. https://doi.org/10.1186/s12911- 020-01297-6

    2020 doi

  44. [53]

    Knowledge discovery in heterogeneous and unstructured data of industry 4.0 systems: Challenges and approaches

    Kamm S, Jazdi N, Weyrich M. Knowledge discovery in heterogeneous and unstructured data of industry 4.0 systems: Challenges and approaches. Procedia CIRP . 2021;104:975- 80. https://doi.org/10.1016/j.procir.2021.11.164

    2021 doi

  45. [54]

    Big data analytics in oil and gas industry: An emerging trend

    Mohammadpoor M, Torabi F. Big data analytics in oil and gas industry: An emerging trend. Petroleum. 2020;6(4):321-8. https://doi.org/10.1016/j.petlm.2018.11.001

    2020 doi

  46. [55]

    Data integration for large - scale models of species distributions

    Isaac NJ, Jarzyna MA, Keil P , Dambly LI, Boersch-Supan PH, Browning E, et al. Data integration for large - scale models of species distributions. Trends Ecol Evol. 2020;35(1):56 -67. https://doi.org/10.1016/j.tree.2019.08.006

    2020 doi

  47. [56]

    The use of big data analytics in healthcare

    Batko K, Ślęzak A. The use of big data analytics in healthcare. J Big Data. 2022;9(1):3. https://doi.org/10.1186/s40537-021-00553-4

    2022 doi

  48. [57]

    Cost -effective bad synchrophasor data detection based on unsupervised time -series data analytic

    Zhu L, Hill DJ. Cost -effective bad synchrophasor data detection based on unsupervised time -series data analytic. IEEE Internet Things J. 2020;8(3):2027-39. https://doi.org/10.1109/JIOT .2020.3016032

    2020

  49. [58]

    Crossfun: Multi-v i e w j o i n t c r o s s f u s i o n n e t w o r k f o r ti m e s e r i e s anomaly detection

    Bai Y , Wang J, Zhang X, Miao X, Lin Y . Crossfun: Multi-v i e w j o i n t c r o s s f u s i o n n e t w o r k f o r ti m e s e r i e s anomaly detection. IEEE Trans Instrum Meas. 2023. https://doi.org/10.1109/TIM.2023.3315420

    2023

  50. [59]

    Big data and stream processing platforms for industry 4.0 requirements mapping for a predictive maintenance use case

    Sahal R, Breslin JG, Ali MI. Big data and stream processing platforms for industry 4.0 requirements mapping for a predictive maintenance use case. J Manuf Syst. 2020;54:138 -51. https://doi.org/10.1016/j.jmsy.2019.11.004

    2020 doi

  51. [60]

    Sice: An improved missing data imputation technique

    Khan SI, Hoque ASML. Sice: An improved missing data imputation technique. J Big Data. 2020;7:37. https://doi.org/10.1186/s40537-020-00313-w

    2020 doi

  52. [61]

    Generative adversarial networks for imputing missing data for big data clinical research

    Dong W, Fong DYT, Yoon J-s, Wan EYF, Bedford LE, Tang EHM, et al. Generative adversarial networks for imputing missing data for big data clinical research. BMC Med Res Methodol. 2021;21:1 -10. https://doi.org/10.1186/s12874-021-01272-3. 59

    2021 doi

  53. [62]

    Statistical analysis with missing data: John Wiley & Sons; 2019

    Little RJ, Rubin DB. Statistical analysis with missing data: John Wiley & Sons; 2019

    2019

  54. [63]

    Load image inpainting: An improved u-net based load missing data recovery method

    Liu L, Liu Y . Load image inpainting: An improved u-net based load missing data recovery method. Appl Energy. 2022;327:119988. https://doi.org/10.1016/j.apenergy.2022.119988

    2022

  55. [64]

    Fundamentals of machine learning for predictive data analytics: Algorithms, worked examples, and case studies: MIT press; 2020

    Kelleher JD, Mac Namee B, D'arcy A. Fundamentals of machine learning for predictive data analytics: Algorithms, worked examples, and case studies: MIT press; 2020

    2020

  56. [65]

    Characteristic-based clustering for time series data

    Wang X, Smith K, Hyndman R. Characteristic-based clustering for time series data. Data Min Knowl Discov. 2006;13:335-64. https://doi.org/10.1007/s10618-005-0039-x

    2006 doi

  57. [66]

    Gratis: Generating time series with diverse and controllable characteristics

    Kang Y , Hyndman RJ, Li F. Gratis: Generating time series with diverse and controllable characteristics. Stat Anal Data Min. 2020;13(4):354-76. https://doi.org/10.1002/sam.11461

    2020 doi

  58. [67]

    A review on outlier/anomaly detection in time series data

    Blázquez-García A, Conde A, Mori U, Lozano JA. A review on outlier/anomaly detection in time series data. ACM Comput Surv. 2021;54(3):1-33. https://doi.org/10.1145/3444690

    2021 doi

  59. [68]

    Anomaly detection in time series: A comprehensive evaluation

    Schmidl S, Wenig P , Papenbrock T. Anomaly detection in time series: A comprehensive evaluation. Proceedings of the VLDB Endowment. 2022;15(9):1779-97. https://doi.org/10.14778/3538598.3538602

    2022

  60. [69]

    Understanding and using time series analyses in addiction research

    Beard E, Marsden J, Brown J, Tombor I, Stapleton J, Michie S, et al. Understanding and using time series analyses in addiction research. Addiction. 2019;114(10):1866-84. https://doi.org/10.1111/add.14643

    2019 doi

  61. [70]

    Framework and modelling of inclusive manufacturing system

    Singh S, Mahanty B, Tiwari MK. Framework and modelling of inclusive manufacturing system. Int J Comput Integ M. 2019;32(2):105-23. https://doi.org/10.1080/0951192x.2018.1550678

    2019

  62. [71]

    Challenges and trends of big data analytics

    Li H, Lü X. Challenges and trends of big data analytics. 2014 Ninth International Conference on P2P , Parallel, Grid, Cloud and Internet Computing. 2014:566-7. https://doi.org/10.1109/3pgcic.2014.136

    2014 doi

  63. [72]

    Initiating predictive maintenance for a conveyor motor in a bottling plant using industry 4.0 concepts

    Kiangala KS, Wang ZH. Initiating predictive maintenance for a conveyor motor in a bottling plant using industry 4.0 concepts. Int J Adv Manuf Tech. 2018;97:3251 -71. https://doi.org/10.1007/s00170-018- 2093-8

    2018 doi

  64. [73]

    Intelligent predictive maintenance for fault diagnosis and prognosis in machine centers: Industry 4.0 scenario

    Li Z, Wang Y , Wang KS. Intelligent predictive maintenance for fault diagnosis and prognosis in machine centers: Industry 4.0 scenario. Adv Manuf. 2017;5(4):377-87. https://doi.org/10.1007/s40436-017-0203- 8. 60

    2017 doi

  65. [74]

    Missing data imputation techniques

    Song Q, Shepperd M. Missing data imputation techniques. Int J Bus Intell Data Min. 2007;2(3):261 -91. https://doi.org/10.1504/IJBIDM.2007.015485

    2007 arXiv

  66. [75]

    Missing value imputation in multivariate time series with end-to-end generative adversarial networks

    Zhang Y , Zhou B, Cai X, Guo W, Ding X, Yuan X. Missing value imputation in multivariate time series with end-to-end generative adversarial networks. Inf Sci. 2021;551:67 -82. https://doi.org/10.1016/j.ins.2020.11.035

    2021 doi

  67. [76]

    A novel hybrid feature importance and feature interaction detection framework for predictive optimization in industry 4.0 applications

    Ma Z, Jørgensen B, Ma ZG. A novel hybrid feature importance and feature interaction detection framework for predictive optimization in industry 4.0 applications. IECON 2023–49th Annual Conference of the IEEE Industrial Electronics Society; 2023; Singapore: IEEE. https://doi.or...

    2023

  68. [77]

    A data-driven two-phase multi-split causal ensemble model for time series

    Ma Z, Kemmerling M, Buschmann D, Enslin C, Lütticke D, Schmitt RH. A data-driven two-phase multi-split causal ensemble model for time series. Symmetry. 2023;15:982. https://doi.org/10.3390/sym15050982

    2023 doi

  69. [78]

    Statsmodels: Econometric and statistical modeling with python

    Seabold S, Perktold J. Statsmodels: Econometric and statistical modeling with python. Proceedings of the 9th Python in Science Conference; 2010: Austin, TX. https://doi.org/10.25080/Majora-92bf1922-011

    2010 doi

  70. [93]

    https://doi.org/10.1007/s40747-021-00637-x

    doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.