Data Science Tech Brief By HackerNoon

Data Science Tech Brief By HackerNoon

By HackerNoonNewsTech News
Download on the App Store

Data Science Tech Brief By HackerNoon episodes

  • Data Science Interview Question: Creating ROC & Precision Recall Curves From Scratch

    This story was originally published on HackerNoon at: https://hackernoon.com/data-science-interview-question-creating-roc-and-precision-recall-curves-from-scratch.


    This is one of the popular data science interview questions which requires one to create the ROC and similar curves from scratch.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data-science, #data-science-interview, #precision-and-recall, #precision-recall-curves, #roc-data-science, #data-analysis, #data-science-job-questions, #hackernoon-top-story, and more.


    This story was written by: @varunnakra1. Learn more about this writer by checking @varunnakra1's about page,
    and for more stories, please visit hackernoon.com.


    This is one of the popular data science interview questions which requires one to create the ROC and similar curves from scratch. For the purposes of this story, I will assume that readers are aware of the meaning and the calculations behind these metrics and what they represent and how are they interpreted. We start with importing the necessary libraries (we import math as well because that module is used in calculations)

    9 min
  • Why Should Companies Outsource Data Processing?

    This story was originally published on HackerNoon at: https://hackernoon.com/why-should-companies-outsource-data-processing.


    Data processing outsourcing boosts efficiency, reduces costs, and enhances decision-making, helping businesses manage and leverage vast data effectively.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data, #data-processing, #data-outsourcing, #data-mangagement, #data-security, #data-cost-reduction, #data-efficiency, #data-science, and more.


    This story was written by: @rayanpotterr. Learn more about this writer by checking @rayanpotterr's about page,
    and for more stories, please visit hackernoon.com.


    Data processing is an essential business process and it consists of activities like order processing, form processing, compilation of mailing lists, and processing of different organizational and business information. Data processing outsourcing also offers a two-fold benefit i.e. low operational expenses and increased operational efficiency. It also helps in enhancing data quality, obtaining insights quicker to make well-informed and timely business decisions.

    7 min
  • The Role of Big Data in Developing New Medicines

    This story was originally published on HackerNoon at: https://hackernoon.com/the-role-of-big-data-in-developing-new-medicines.


    Drug development is one of the most crucial — and time-consuming — processes in medicine. Here's how big data can help.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #big-data, #drug-discovery, #medicine, #drug-development, #artificial-intelligence, #healthcare, #pharmaceutical, #hackernoon-top-story, and more.


    This story was written by: @zacamos. Learn more about this writer by checking @zacamos's about page,
    and for more stories, please visit hackernoon.com.


    Developing a new medicine takes an average of 12 years, but big data can improve every stage of the process. It helps fuel AI drug discovery, identify underserved needs, streamline clinical trials, and monitor for potential issues.

    7 min
  • Building CI Pipeline with Databricks Asset Bundle and GitLab

    This story was originally published on HackerNoon at: https://hackernoon.com/building-ci-pipeline-with-databricks-asset-bundle-and-gitlab.


    Databricks Asset Bundle streamlines the development of complex data, analytics, and ML projects for the Databricks platform.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #databricks, #gitlab, #devops, #mlops-platforms, #databricks-asset-bundles, #databricks-gui, #how-to-build-a-ci-pipeline, #hackernoon-top-story, and more.


    This story was written by: @neshom. Learn more about this writer by checking @neshom's about page,
    and for more stories, please visit hackernoon.com.


    In the previous blog, I showed you how to build a CI pipeline using Databricks CLI eXtensions and GitLab. In this post, I will show you how to achieve the same objective with the latest and recommended Databricks deployment framework, Databricks Asset Bundles.

    11 min
  • How I'm Building an AI for Analytics Service

    This story was originally published on HackerNoon at: https://hackernoon.com/how-im-building-an-ai-for-analytics-service.


    In this article I want to share my experience with developing an AI service for a web analytics platform called Swetrix.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data-analysis, #ai, #analytics, #website-traffic, #software-architecture, #machine-learning, #predictive-analytics, #hackernoon-top-story, and more.


    This story was written by: @pro1code1hack. Learn more about this writer by checking @pro1code1hack's about page,
    and for more stories, please visit hackernoon.com.


    In this article I want to share my experience with developing an AI service for a web analytics platform, called Swetrix. My aim was to develop a machine learning model that would predict future website traffic based on the data displayed on the following screenshot. The end goal is to have a clear vision for the customer of what traffic will appear on their website in the future.

    8 min
  • Real-Time Anomaly Detection in Underwater Gliders: Experimental Evaluation

    This story was originally published on HackerNoon at: https://hackernoon.com/real-time-anomaly-detection-in-underwater-gliders-experimental-evaluation.


    This paper presents a real-time anomaly detection algorithm to enhance underwater glider safety using datasets from actual deployments.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data-analysis, #machine-learning, #underwater-gliders, #anomaly-detection, #oceanography, #glider-navigation, #ocean-data, #marine-robotics, and more.


    This story was written by: @oceanography. Learn more about this writer by checking @oceanography's about page,
    and for more stories, please visit hackernoon.com.


    We apply the anomaly detection algorithm to four glider deployments across the coastal ocean of Florida and Georgia, USA. For evaluation, the anomaly detected by the algorithm is cross-validated by high-resolution glider DBD data and pilot notes. We simulate the online detection process on SBD and compare the result with that detected from DBD.

    11 min
  • Real-Time Anomaly Detection in Underwater Gliders: Abstract and Intro

    This story was originally published on HackerNoon at: https://hackernoon.com/real-time-anomaly-detection-in-underwater-gliders-abstract-and-intro.


    This paper presents a real-time anomaly detection algorithm to enhance underwater glider safety, using datasets from actual deployments.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data-analysis, #machine-learning, #underwater-gliders, #anomaly-detection, #oceanography, #glider-navigation, #ocean-data, #marine-robotics, and more.


    This story was written by: @oceanography. Learn more about this writer by checking @oceanography's about page,
    and for more stories, please visit hackernoon.com.


    Underwater gliders are widely used in oceanography for a range of applications. However, unpredictable events like shark strikes or remora attachments can lead to abnormal glider behavior or even loss of the instrument. This paper employs an anomaly detection algorithm to assess operational conditions of underwater gliders in the real-world ocean environment. Prompt alerts are provided to glider pilots upon detecting any anomaly.

    8 min
  • The Power of Universal Semantic Layers: Insights from Cube Co-founder Artyom Keydunov

    This story was originally published on HackerNoon at: https://hackernoon.com/the-power-of-universal-semantic-layers-insights-from-cube-co-founder-artyom-keydunov.


    What is a universal semantic layer, and how is it different from a semantic layer? Is there actual semantics involved? Who uses that, how, and what for?
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #analytics, #business-intelligence, #data-modeling, #data-integration, #knowledge-graph, #universal-semantic-layer, #data-visualization, #data-warehouses, and more.


    This story was written by: @linked_do. Learn more about this writer by checking @linked_do's about page,
    and for more stories, please visit hackernoon.com.


    What is a universal semantic layer, and how is it different from a semantic layer? Is there actual semantics involved? Who uses that, how, and what for?

    10 min
  • A Comprehensive Guide to Building DolphinScheduler 3.2.0 Production-Grade Cluster Deployment

    This story was originally published on HackerNoon at: https://hackernoon.com/a-comprehensive-guide-to-building-dolphinscheduler-320-production-grade-cluster-deployment.


    In version 3.2.0, DolphinScheduler introduces a series of new features and improvements, significantly enhancing its stability.
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #data-science, #workflow-management, #opensource, #programming, #dolphinscheduler, #apache-dolphinscheduler, #big-data, #cluster-deployment, and more.


    This story was written by: @zhoujieguang. Learn more about this writer by checking @zhoujieguang's about page,
    and for more stories, please visit hackernoon.com.


    DolphinScheduler provides powerful workflow management and scheduling capabilities for data engineers by simplifying complex task dependencies. In version 3.2.0, DolphinScheduler introduces a series of new features and improvements, significantly enhancing its stability and availability in production environments.

    10 min
  • Why Monitoring a Distributed Database is More Complex Than You Might Expect

    This story was originally published on HackerNoon at: https://hackernoon.com/why-monitoring-a-distributed-database-more-complex-than-you-might-expect.


    Why is monitoring a distributed database more complex than you might expect
    Check more stories related to data-science at: https://hackernoon.com/c/data-science.
    You can also check exclusive content about #distributed-databases, #opentelemetry, #monitor-distributed-databases, #apache-ignite, #monitoring-systems, #database-monitoring-challenges, #data-acquisition-models, #push-vs-pull-acquisition, and more.


    This story was written by: @ingvard. Learn more about this writer by checking @ingvard's about page,
    and for more stories, please visit hackernoon.com.


    In this article, we will dive into the complexities of monitoring distributed databases from the perspective of a monitoring system developer. I will try to cover the following topics: managing multiple nodes, network restrictions, and issues related to high throughput caused by a large number of metrics.

    21 min

About Data Science Tech Brief By HackerNoon

From the publisher's feed

Learn the latest data science updates in the tech world.