Trending December 2023 # What Is Clustering In Data Mining? # Suggested January 2024 # Top 13 Popular

You are reading the article What Is Clustering In Data Mining? updated in December 2023 on the website Kientrucdochoi.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested January 2024 What Is Clustering In Data Mining?

Introduction to Data Mining

This is a data mining method used to place data elements in their similar groups. Cluster is the procedure of dividing data objects into subclasses. Clustering quality depends on the way that we used. Clustering is also called data segmentation as large data groups are divided by their similarity.

Start Your Free Data Science Course

What is Clustering in Data Mining?

Clustering is the grouping of specific objects based on their characteristics and their similarities. As for data mining, this methodology divides the data that is best suited to the desired analysis using a special join algorithm. This analysis allows an object not to be part or strictly part of a cluster, which is called the hard partitioning of this type. However, smooth partitions suggest that each object in the same degree belongs to a cluster. More specific divisions can be created like objects of multiple clusters, a single cluster can be forced to participate, or even hierarchic trees can be constructed in group relations. This filesystem can be put into place in different ways based on various models. These Distinct Algorithms apply to each and every model, distinguishing their properties as well as their results. A good clustering algorithm is able to identify the cluster independent of cluster shape. There are 3 basic stages of clustering algorithm which are shown below

Clustering Algorithms in Data Mining Methods of Clustering in Data Mining

The different methods of clustering in data mining are as explained below:

Partitioning based Method

Density-based Method

Centroid-based Method

Hierarchical Method

Grid-Based Method

Model-Based Method

1. Partitioning based Method

The partition algorithm divides data into many subsets.

Let’s assume the partitioning algorithm builds a partition of data and n objects present in the database. Hence each section will be represented ask ≤ n.

This gives an idea that the classification of the data is in k groups, which can be shown below

Figure 1 shows original points in clustering

Figure 2 shows Partition clustering after applying an algorithm

This indicates that each group has at least one object, and every object, must belong to exactly one group.

2. Density-Based Method

These algorithms produce clusters in a determined location based on the high density of data set participants. It aggregates some range notion for group members in clusters to a density standard level. Such processes can perform less in detecting the group’s Surface areas.

3. Centroid-based Method

A vector of values references almost every cluster in this type of os grouping technique. Compared to other groups, each object is part of the group with a minimum difference in value. The number of groups should be predefined, which is the most significant algorithm problem of this type. This methodology is the closest to the subject of identification and is widely used for problems of optimization.

4. Hierarchical Method

The method will create a hierarchical decomposition of a given set of data objects. Based on how the hierarchical decomposition is formed, we can classify hierarchical methods. This method is shown as follows

Agglomerative Approach

Divisive Approach

Divisive Approach is also known as the Top-Down Approach. We begin with all the things in the same cluster. This method is rigid, i.e., it can never be undone once a fusion or division is completed

5. Grid-Based Method

Advantages of Hierarchical Clustering are as follows

It applies to any attribute type.

It provides flexibility related to the level of granularity.

6. Model-Based Method

This method uses a hypothesized model based on probability distribution. By clustering the density function, this method locates the clusters. It reflects the data points’ spatial distribution.

Application of clustering in Data Mining Conclusion

Clustering is important in data mining and its analysis. In this article, we have seen how clustering can be done by applying various clustering algorithms and its application in real life.

Difference Between Data Mining And Web Mining?

Data mining is the procedure of exploration and analysis of huge quantities of data to find meaningful patterns and rules. On the other hand, web mining defines the process of using data mining techniques to extract useful data patterns and trends from web-based records and services, server logs, and hyperlinks.

Read this article to learn more about Data Mining and Web Mining and how they are different from each other.

What is Data Mining?

Data mining is the process of discovering meaningful new correlations, patterns, and trends by shifting through a large amount of data stored in repositories, using pattern recognition technologies as well as statistical and mathematical techniques. It is the analysis of observational datasets to find unsuspected relationships and to summarize the data in novel ways that are both understandable and beneficial to the data owner.

Data mining is the process of selection, exploration, and modeling of large quantities of data to discover regularities or relations that are at first unknown to obtain clear and useful results for the owner of the database. Data mining is the procedure of exploration and analysis by automatic or semi-automatic defines of huge quantities of data to find meaningful patterns and rules.

Data Mining is similar to Data Science. It is carried out by a person, in a specific situation, on a particular data set, with an objective. This process includes various types of services such as text mining, web mining, audio and video mining, pictorial data mining, and social media mining. It is completed through software that is simple or hugely specific.

By outsourcing data mining, all the work can be completed quicker with low operation costs. Specialized firms can also use new technologies to set data that is impossible to situate manually. There are tons of information available on various platforms, but very little knowledge is accessible.

The biggest challenge is to analyze the data to extract important information that can be used to solve a problem or for company development. There are many powerful instruments and techniques available to mine data and find better insight from it.

What is Web Mining?

Web mining defines the process of using data mining techniques to extract beneficial patterns trends and data generally with the help of the web by dealing with it from web-based records and services, server logs, and hyperlinks. The main goal of web mining is to find the designs in web data by collecting and analyzing data to get important insights.

Web mining can broadly be viewed as the application of adapted data mining techniques to the internet, whereas data mining is represented as the application of the algorithm to find patterns on mostly structured data fixed into a knowledge discovery process.

Web mining has distinctive features to offer a set of multiple data types. The web has several aspects that yield multiple approaches for the mining process, including web pages including text, web pages are connected via hyperlinks, and user activity can be monitored via web server logs.

Difference between Data Mining and Web Mining

The following table highlights all the major differences between data mining and web mining −

S.No.

Data Mining

Web Mining

The main goal of data mining is the analysis of observational datasets to find unsuspected relationships and to summarize the data in novel ways that are both understandable and beneficial to the data owner.

The main goal of web mining is to find the designs in web data by collecting and analyzing data to get important insights.

It is the process of selection, exploration, and modeling of large quantities of data to discover regularities or relations that are at first unknown to obtain clear and useful results for the owner of the database.

Web mining can broadly be viewed as the application of adapted data mining techniques to the internet.

This process includes various types of services such as text mining, web mining, audio and video mining, pictorial data mining, and social media mining.

The web has several aspects that yield multiple approaches for the mining process, including web pages including text, web pages are connected via hyperlinks, and user activity can be monitored via web server logs.

Data mining uses business intelligent applications that includes information to improve business activities.

Web mining uses data analytics for medication of raw data into a meaningful format.

The target users of data mining include data engineers and data scientists.

The target users of web mining include data analysts or web analysts.

Data mining uses tools like machine learning.

Web mining uses tools like PageRank, Apache logs, Scrapy, etc.

For decision-making process, many organizations use data mining results.

Web mining is significant for pulling the existing data mining process.

Conclusion

To conclude, the main objective of data mining is to analyze observational datasets to find unsuspected relationships and to summarize the data in novel ways that are both understandable and beneficial to the data owner. In contrast, the primary objective of web mining is to find the designs in web data by collecting and analyzing data to get important insights.

What, Why And How Of Spectral Clustering!

This article was published as a part of the Data Science Blogathon

When given a bunch of movies to organize, you might sort them based on similarity measures such as genre, year of release, casting, revenue generated, and so on. While you categorize them based on a measure, it doesn’t mean that a measure used by others is of no good. Our perception of data is greatly influenced by the clustering measure being used. When you believe that two sci-fi movies belong to the same cluster, someone else might consider two movies released in the same year to be in the same cluster. Because you might have used the genre as a measure to cluster and the other person might have used the year of release as a measure.

Classifications vs Clustering

As humans, in machine learning, a widely used unsupervised algorithm to group unlabeled data points by similarity and distance measures is clustering. If the data points are labeled, grouping is known as classification. Clustering algorithms have their application in many places including anomaly detection, image segmentation, search result grouping, market segmentation, and social network analysis. Clustering is one of the initial steps done in exploratory data analysis to visualize the similarity and to identify the pattern lying hidden in data points. The motive of clustering is to find the similarity within a cluster and the difference between two clusters.

Support Vector Machines, Decision trees, Random forests, Linear classifiers and Neural networks are few classification algorithms whereas K means clustering, Fuzzy analysis clustering, Mean shift, DBSCAN and Spectral are clustering algorithms.

There are two major approaches in clustering. They are:

Compactness

Connectivity

In compactness, the points are closer to each other and are compact towards the cluster center. Distance is used as a measure to compute closeness. There are different types of distance metrics that are in use. A few of them are Euclidean distance, Manhattan distance, Minkowski distance, and Hamming distance. K-means algorithm uses the compactness approach. In connectivity, the points in a cluster are either immediately next to each other (epsilon distance) or connected. Even if the distance is less, they are not put in the same cluster. Spectral clustering is one of the techniques to follow this approach.

How to do Spectral Clustering?

The three major steps involved in spectral clustering are: constructing a similarity graph, projecting data onto a lower-dimensional space, and clustering the data. Given a set of points S in a higher-dimensional space, it can be elaborated as follows:

1. Form a distance matrix

2. Transform the distance matrix into an affinity matrix A

3. Compute the degree matrix D and the Laplacian matrix L = D – A.

4. Find the eigenvalues and eigenvectors of L.

5. With the eigenvectors of k largest eigenvalues computed from the previous step form a matrix.

6. Normalize the vectors.

7. Cluster the data points in k-dimensional space.

Pros and Cons of Spectral Clustering

Spectral clustering helps us overcome two major problems in clustering: one being the shape of the cluster and the other is determining the cluster centroid. K-means algorithm generally assumes that the clusters are spherical or round i.e. within k-radius from the cluster centroid. In K means, many iterations are required to determine the cluster centroid. In spectral, the clusters do not follow a fixed shape or pattern. Points that are far away but connected belong to the same cluster and the points which are less distant from each other could belong to different clusters if they are not connected. This implies that the algorithm could be effective for data of different shapes and sizes.

When compared with other algorithms, it is computationally fast for sparse datasets of several thousand data points. You don’t need the actual dataset to work with. Distance or Though it might be costly to compute for large datasets as eigenvalues and eigenvectors need to be computed and then clustering is to be done. But the algorithms try to cut the cost. The number of clusters (k) needs to be fixed before starting the procedure.

Where can I use spectral clustering?

Spectral clustering has its application in many areas which includes: image segmentation, educational data mining, entity resolution, speech separation, spectral clustering of protein sequences, text image segmentation. Though spectral clustering is a technique based on graph theory, the approach is used to identify communities of vertices in a graph based on the edges connecting them. This method is flexible and allows us to cluster non-graph data as well either with or without the original data.

Python Code:

where X is a NumPy array of shape (n_samples, n_features) and the returned labels is a NumPy array of shape (n_samples, ).

Clustering with the spectral clustering and visualizing the data sc=SpectralClustering(n_clusters=4).fit(x) SpectralClustering(affinity='rbf', assign_labels='kmeans', coef0=1, degree=3, eigen_solver=None, eigen_tol=0.0, gamma=1.0, kernel_params=None, n_clusters=4, n_components=None, n_init=10, n_jobs=None, n_neighbors=10, random_state=None) labels = sc.labels_ plt.scatter(x[:,0], x[:,1], c=labels) plt.show() print(sc)

Let’s try changing the number of clusters. f = plt.figure() f.add_subplot(2, 2, 1) for i in range(2, 6): sc = SpectralClustering(n_clusters=i).fit(x) f.add_subplot(2, 2, i-1) plt.scatter(x[:,0], x[:,1], s=5, c=sc.labels_, label="n_cluster-"+str(i)) plt.legend() plt.show()

I’m Keerthana, a data science student fascinated by Math and its applications in other domains. I’m also interested in writing Math and Data Science related articles. You can connect with me on LinkedIn and Instagram. Check out my other articles here.

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Clustering In Power Bi And Python: How It Works

Below are two visuals with clusters created in Power BI. The one on the left is a table and the other on the right is a scatter plot.

Our scatter plot has two-dimensional clustering, using two data sets to create clusters. The first is the shopping data set, consisting of customer ID, annual income, and age, and the other is the spending score. Meanwhile, our table uses multi-dimensional clustering, which uses all the data sets.

To demonstrate how it works, I will need to eliminate the clusters so we can start with each visual from scratch. Once you create these clusters in Power BI, they become available as little parameters or dimensions in your data set.

We’ll delete the multi-dimensional clusters using this process and then get our table and scatter plot back, starting with the latter.

If we choose Age and Spending Score from our data set, Power BI will automatically summarize them into two dimensions inside our scatter plot.

If we add our Customer ID to our Values by dragging it from the Fields section to the Values section, we will get that scatter plot back, just like in the image below.

In the Clusters window, we can enter a Name for our clusters, select a Field, write a Description, and choose the Number of Clusters.

We will name our clusters Shopping Score Age, select CustomerID for the field, and input Clusters for CustomerID as a description. We’ll then set the number of clusters to Auto.

The current dimensions in our table, which you can find in the column headers, are Customer ID, Annual Income, Age, and Spending Score. A dimension we didn’t bring in is Gender.

Let’s bring this dimension into our table and scatter plot by dragging it from the Fields section to the Values section, as we did when we added our Customer ID.

As you can see above, we now have a Gender dimension that indicates whether the person is Male or Female. However, if we go to Automatically find clusters to create a cluster for this dimension, it will result in a “Number of fields exceeded” response.

There are two things we can do to go around this roadblock. We can turn the variables, Male and Female, into 0 and 1, giving them numerical values, or we can remove them. However, removing them means that this dimension will no longer be part of our clustering consideration.

Let’s try the second method and remove Gender by unselecting or unchecking it in the Fields section. We then go to our ellipses and select Automatically find clusters.

Now let’s proceed on how to cluster using Python, where we’ll run across the data and create a new data set. We’ll be using an unsupervised machine-learning model that will give you similar results for your multidimensional clustering. I will also show you how to put different algorithms and tweak them along the way.

We first need to run a Command Prompt, like Anaconda Prompt that we’ll be using in this example, and install a package called PyCaret here. After opening the Anaconda prompt, we enter pip install pycaret to install the package.

We’ll put that machine learning algorithm into our Python Script using a simple code. We start by entering from pycaret.clustering import * to import our package. We then type in dataset = get_clusters() in the next line to replace the data set and bring in the function called get_clusters.

We want our function to get our data set, so we’ll assign it with a variable by entering data set = inside the open and close parenthesis. Next, we add our model called K-Means and assign the number of clusters for our model.

Before we run our Python script, first let me show you the different models we use in PyCaret. As you can see below, we’re using K-Means, which follows the same logic as having that Center Point. Aside from that, we also have kmodes, a similar type of clustering.

These other clustering models above will work based on your needs and are much more flexible and not blob-based. If you have a different data set and feel like your Power BI model isn’t working, you can use all of these models. You can go to the Python script section highlighted below and specify the one you want.

Now we can run our Python script using the K-means unsupervised machine learning algorithm. As you get new data, K-means will learn and alter those Center Points and give you better clustering.

Python allows you to assign better names for your clusters to make them more digestible to your users, a feature absent when clustering in Power BI.

What Is Growth In Digital Marketing?

Since business gets immense response through online marketing activities, the bar graph of digital marketing never slows down. As online users have been growing, so have the opportunities for digital marketing. However, a few more factors contribute to its growth aspect. If you’re planning to start digital marketing and looking for the answer whether growth in digital marketing, read the post to get the input.

Growth in any sector depends on its impact on business and how the revenue is generated using it. In this light, digital marketing is promising marketing with good growth and opportunities for business owners to earn good revenue using various marketing channels.

Before going for a growth discussion, let’s understand digital marketing!

Digital Marketing: A Brief Types of Digital Marketing

Digital marketing is diverse. Therefore, marketers can use several marketing channels and experiment with all possibilities of business opportunities. Among many modules, 8 primary modules are immensely popular in promoting the business online, which include −

Search Engine Optimization

Search engine optimization, or SEO, is an essential marketing channel that marketers depend largely on for Google ranking. There are several measures for SEO like keywords metrics, link building, Google SERP, snippets, etc. Marketers take care of On-Page SEO and Off-Page SEO for the better rank of business websites and analyze the traffic. Thus, the search optimization deal with Google ranking.

Content Marketing

It’s the most important marketing channel in text, infographics, and video formats. Relevant content based on the product/service can attract potential buyers. Marketers channelize content based on the users’ preferences and monitor the dashboard. Video content and infographics attract more traffic than text format. Because visual and audio content can impact human brains more. Thus, content marketing serves a great purpose in digital marketing.

Email Marketing

Through emails, business owners can send millions of emails once at a time. With relevant content and a proper audience, email marketing would be a great marketing tool.

Marketing Analytics

Marketing analytics deal with the data generated from various digital platforms. Analytics experts analyze the data and channel further marketing approaches.

Social Media Marketing

Social media once was the platform to connect with distant people and build networks. But now, the platforms come with massive business opportunities that marketers channelize them through different formats. With the increased number of opportunities, digital marketers utilize them successfully.

Affiliate Marketing

Affiliate marketing promotes other products from your account or page and earns a good commission. This way, businesses do sales by building several networks.

Mobile Marketing

When business websites become optimized for mobile, mobile marketing channels open the doors to generating traffic. As smartphone users are more than laptops or desktops, digital marketers can bring outstanding results.

The Current Trend of Digital Marketing: New Updates of Various Marketing Channels

With the emergence of technology, automation, and artificial intelligence, digital marketing is getting a new shape with a more creative approach. Several tools, applications, and software are available in the market to ease the workload of marketers. The benefit is that marketers can now try a more practical experiment with the marketing channels. However, the current trend is also more challenging than before, but it’ll never impact the business; marketers need to be more innovative in using modern science in digital marketing.

The recent trend introduces many other marketing channels, so businesses should not go with no stone unturned. As the need grows, digital marketing agencies and marketers are seeding the growth prospect for the coming days. Below is the list of current digital marketing trends that can bring positive results −

Voice Search Optimization

ChatBots

Artificial Intelligence(AI)

Augmented Reality(AR)

Personalized Automated Email Marketing

Micro-Macro Influencers

Video Marketing

Instagram Reels Marketing

YouTube Shorts

User Generated Content

The Growth Prospects of Digital Marketing

Digital marketing depends primarily on a creative skill set, a marketing enthusiast, and a good knowledge of digitalization. When all these align, the learners must add the zeal to learn a new skill. Thus, digital marketing introduces a new and challenging skill that newcomers and experienced marketers enjoy doing marketing digitally.

The number of internet users has been skyrocketing (in 2023, it was 622 active internet users), which will be 900 million by 2025. The data indicates how digital marketing will boom in the upcoming days to ensure its future growth.

These humongous internet users will secure business possibilities and marketing growth. However, there are a few more metrics that depend largely on the growth prospects, which include −

Easy Available Data

In any business, data plays a crucial role. Digital marketing is no exception. Marketers can collect data using social media, video marketing, podcast, subscription to newsletters, etc. Marketers can utilize the data for product promotion, branding, email marketing, etc.; thus, data availability adds growth to digital marketing approaches.

Online Presence of Business Social Media Addiction Usage of AI No Barriers Conclusion

The growth of digital marketing has started peaking with the help of new technology, automation, AI, and brilliant creative marketers. Therefore, the number of digital marketing agencies has been increasing, which ensures the growth of digital marketing activities that blow the digital surface with new challenges and positive results to the clients’ table. A creative mindset, innovative technology, and positive results also contribute largely to its growth model.

What Is A Bug In Software Testing?

Introduction to Bug in Software Testing

Web development, programming languages, Software testing & others

Life Cycle of Bug in Software Testing

The Bug Life cycle is also known as a Defect Life cycle. It is a phase of a defect that occupies the different states during its lifetime. It starts when a testing device finds a new defect and ends when the testing device removes that defect and it is ensured that the defect is not replicated. It is now time to understand, through a basic diagram as shown below, the true workflow of a defect life cycle.

Below is the Diagram of the Bug life cycle:

Status of Bug

Let us see each component of the bug life cycle.

1. Open

The programmer begins the bugs analysis process here, where possible, and works to repair it. If the programmer thinks that the defect is not sufficient, then an error depending on the particular reason may be passed to the following four states, Reject or Not, namely Duplicate.

2. New 3. Assigned

The development team is allocated a newly created fault for operating on the fault at this level. This will be delegated to a designer by the project leader or the team’s boss.

4. Pending Retest

Upon fixing the defect, the designer will give the tester the fault for retesting the fault and the state of the defect stay in pending re-test ‘ until the tester works on re-testing the fault.

5. Fixed

If the developer completes the task of repairing a defect by making the necessary changes, the defect status can be called “Fixed.”

6. Verified

If the tester has no problem with the defect after the designer has been assigned the defect to the testing device and thought that if it was correctly repaired, the defect status is assigned “confirmed”.

7. Re-open 8. Closed

If the defect is absent, the tester changes the defect status to ‘Closed’.

9. Retest

The tester then begins the task of re-testing the defect to check whether the defect is correctly fixed by the developer as required by the requirement.

10. Duplicate

If the developer considers the defect similar to any other defect, or if the defect definition blends into any other defect, the defect status is changed by the developer to ‘duplicate’.

Parameter of Bug in Software Testing

Date of issue, approvals, author, and status.

Severity and incident priority.

The test case showed the problem.

Incident definition with reproductive steps.

Guidance for Deficiency Life Cycle Implementation

The entire team must understand clearly the different conditions of a bug before beginning the research on the defect life cycle.

To prevent confusion in the future, the defect life cycle should be documented properly.

Ensure that every person with any task related to the Default Life Cycle understands his / her responsibility for better results very clearly.

Every individual who changes the status of a defect should know the status properly which should provide enough information about the status of a defect and the reason for it so that everybody who works on that defect can easily see the reason for the defect.

The defect tracking tool should be handled with care in the workflow of the defect life cycle to ensure consistency between the defects.

Conclusion

I hope you’ve got some knowledge of a defect’s life cycle. This article will also assist you conveniently in the future if you deal with software defects.