Introduction

Docker has become an essential tool for modern software development and deployment. It enables developers to package their applications and dependencies into portable containers, making it easier to build, ship, and run applications across different environments. Docker also provides several benefits, including consistency, portability, scalability, and isolation. In this article, we will explore Docker and Docker Compose and their use cases, as well as discuss why Docker is important.

Docker

Docker is a platform for building, shipping, and running applications in containers. A container is a lightweight and standalone executable package that contains all the necessary software components, including the application code, dependencies, libraries, and runtime. Containers are isolated from each other and from the host system, which makes them secure and portable. Docker provides several benefits, including:

• Consistency: Docker ensures that the application runs the same way on any environment, whether it's a developer's machine or a production server.

• Portability: Docker containers can be moved between different environments, including development, testing, staging, and production, without any changes to the application code.

• Scalability: Docker enables developers to scale their applications horizontally by adding more containers to handle increased traffic or load.

• Isolation: Docker containers are isolated from each other and from the host system, which makes them secure and reliable.

Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. It enables developers to define the services, networks, and volumes required by their applications in a YAML file, which can be versioned and shared with the team. Docker Compose also provides several benefits, including:

• Simplicity: Docker Compose simplifies the deployment of multi-container applications by defining the dependencies and relationships between the services.

• Modularity: Docker Compose enables developers to break down their applications into smaller, modular components, which can be tested and deployed independently.

• Reproducibility: Docker Compose ensures that the same set of services and dependencies are used across different environments, making it easier to reproduce issues and debug them.

Use Cases

Docker and Docker Compose can be used in a variety of scenarios, including:

• Development: Docker enables developers to create a consistent and isolated development environment, which can be shared with the team. Docker Compose enables developers to define the services required by their applications, such as a database, cache, or message queue, and run them locally.

• Testing: Docker enables developers to create a consistent and isolated testing environment, which can be used to run automated tests. Docker Compose enables developers to define the services required by the tests and run them in a containerized environment.

• Staging: Docker enables developers to create a consistent and isolated staging environment, which can be used to test the application before it's deployed to production. Docker Compose enables developers to define the services required by the staging environment and run them in a containerized environment.

• Production: Docker enables developers to deploy their applications in a consistent and portable way, making it easier to scale and manage them. Docker Compose enables developers to define the services required by the application in production and manage them using an orchestration tool, such as Docker Swarm or Kubernetes.

Volumes

Volumes are a way to persist data between Docker container runs. They are used to store data that needs to survive container restarts or to share data between containers. Volumes can be created and managed using Docker commands or Docker Compose. For example, the following Docker Compose file defines a volume for a database container:

version: '3'

services:
  db:
    image: mysql
    volumes:
      - db-data:/var/lib/mysql

volumes:
  db-data:

In this example, a volume named db-data is defined, and the db service uses it to persist the database data.

Networks

Docker provides several types of networks for container communication, including bridge, host, overlay, and macvlan networks. Bridge networks are the default type and are used to connect containers running on the same Docker host. Host networks allow containers to share the host network namespace, while overlay networks are used for container communication across multiple Docker hosts. Macvlan networks enable containers to have their own MAC address and be treated like physical machines on the network.

Docker Compose enables developers to define networks and connect services to them. For example, the following Docker Compose file defines a bridge network and connects two services to it:

version: '3'

services:
  web:
    image: nginx
    networks:
      - webnet
  db:
    image: mysql
    networks:
      - webnet

networks:
  webnet:

In this example, a bridge network named webnet is defined, and the web and db services are connected to it.

Docker networks allow containers to communicate with each other and with the outside world. By default, containers in the same network can communicate with each other using their container names as hostnames. However, it's also possible to configure Docker networks to provide more granular control over network access.

In a Docker Compose file with multiple modules, it's common to create multiple networks to segregate network access for different projects. For example, you might have a backend that uses a network named "public" to expose APIs to the outside world, and another network named "private" for internal communication with a database. You might also have a frontend module that only needs access to the "public" network.

Here's an example Docker Compose file that sets up multiple networks:

version: '3'

services:
  backend:
    build: .
    networks:
      - public
      - private
    depends_on:
      - db

  db:
    image: postgres
    networks:
      - private

  frontend:
    build: .
    networks:
      - public

networks:
  public:
  private:

In this example, there are three services defined: backend, db, and frontend. The backend and frontend services both use the public network, while the backend and db services both use the private network.

The networks section defines the two networks used by the services. By default, Docker creates a bridge network for each Compose file, but additional networks can be defined using the networks section.

To access a service on a different network, you can specify the network name when referencing the service in your code. For example, if the backend service needs to communicate with the db service, it can use the hostname db and the network name private, like this:

postgres://db:5432/mydatabase

By using multiple networks in your Docker Compose file, you can provide more granular control over network access and better isolate your services.

Services

Services are the building blocks of Docker Compose applications. They represent the containers that make up the application and can be defined with various configuration options, including the image, command, environment variables, ports, volumes, and networks.

For example, the following Docker Compose file defines two services, a web server and a database:

version: '3'

services:
  web:
    image: nginx
    ports:
      - "80:80"
  db:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: password

In this example, the web service uses the nginx image and maps port 80 on the host to port 80 in the container. The db service uses the mysql image and sets the MYSQL_ROOT_PASSWORD environment variable.

Dockerfile and docker-compose.yaml Structure

A Dockerfile is a text file that contains instructions for building a Docker image. Each instruction in the Dockerfile creates a new layer in the image, and the final image is the result of executing all the instructions in order. A typical Dockerfile contains the following sections:

• FROM: specifies the base image for the Docker image.

• ENV: sets environment variables in the container.

• RUN: runs a command inside the container to install packages or configure the environment.

• COPY/ADD: copies files from the host machine to the container.

• CMD/ENTRYPOINT: specifies the command to run when the container starts.

Here is an example Dockerfile for a Node.js application:

FROM node:12

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 3000

CMD [ "npm", "start" ]

In this example, we start with the node:12 base image, set the working directory to /app, copy the package*.json files to the container, run npm install to install dependencies, copy the rest of the application code, expose port 3000, and set the npm start command to run when the container starts.

A docker-compose.yaml file defines a multi-container application, including the services that make up the application, the Dockerfiles used to build the services, and the networks and volumes used by the services. A typical docker-compose.yaml file contains the following sections:

• version: specifies the Docker Compose version.

• services: defines the services that make up the application.

• build: specifies the Dockerfile to use to build each service.

• volumes: defines the volumes used by the services.

• networks: defines the networks used by the services.

Here is an example docker-compose.yaml file for a Node.js application with a web server and a database:

version: '3'

services:
  web:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - .:/app
    depends_on:
      - db
  db:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: password

In this example, we define two services, web and db. The web service uses the Dockerfile in the current directory to build the image, maps port 3000 on the host to port 3000 in the container, mounts the current directory as a volume in the container, and depends on the db service. The db service uses the mysql image and sets the MYSQL_ROOT_PASSWORD environment variable.

DockerFile Best Practices

Dockerfile is a simple text file that contains instructions for building a Docker image, It is an essential component of the Docker ecosystem, and it plays a critical role in creating high-quality images that are easy to manage and deploy. Here are some best practices for writing Dockerfiles:

1. Use a Minimal Base Image: When creating Docker images, it's best to start with a minimal base image. This reduces the size of the image and makes it more efficient to manage. A minimal base image should only include the necessary components required to run your application.

2. Use Layer Caching: Docker images are built using layers. Each instruction in a Dockerfile creates a new layer. When building images, Docker caches layers that have not changed since the last build. By using layer caching, you can speed up the build process and reduce the amount of data that needs to be transferred.

3. Clean Up After Each Step: When creating a Docker image, it's important to remove any files or directories that are no longer needed after each step. This ensures that the image is as small as possible and reduces the risk of security vulnerabilities.

4. Run Containers as Non-Root Users: By default, Docker containers run as the root user. However, this can be a security risk. It's best to run containers as non-root users whenever possible.

5. Use .dockerignore: When building a Docker image, it's important to only include the necessary files and directories. Using .dockerignore can help to exclude unnecessary files and directories from the build context. This reduces the size of the build context and speeds up the build process.

6. Use ENV Instead of ARG: When defining environment variables in a Dockerfile, it's best to use ENV instead of ARG. ENV sets an environment variable that persists in the container, while ARG is only available during the build process.

7. Use LABEL: Docker labels can be used to add metadata to an image. This can be useful for tracking image versions, maintaining an audit trail, and providing additional information about the image.

8. Use HEALTHCHECK: Docker HEALTHCHECK can be used to check the health of a container. This can be useful for detecting and addressing issues before they become critical.

9. Use COPY Instead of ADD: When copying files to a Docker image, it's best to use COPY instead of ADD. COPY only copies the specified files, while ADD can also unpack compressed files, which can be a security risk.

By following these best practices, you can create high-quality Docker images that are efficient, secure, and easy to manage.

Pushing Docker Images to a Private Repository

Docker images can be pushed to a private repository, such as Docker Hub or AWS ECR, for sharing with the team or deployment to production. The following steps demonstrate how to push a Docker image to a private repository and tag and version it:

1. Build the Docker image: docker build -t my-image:v1 .

2. Tag the Docker image with the private repository: `docker tag my-image:v1 my-repo/my-image

1. Login to the private repository: docker login my-repo

2. Push the Docker image to the private repository: docker push my-repo/my-image:v1

Conclusion

In conclusion, Docker has become an essential tool for modern software development and deployment. Its benefits include consistency, portability, scalability, and isolation. With Docker, developers can package their applications and dependencies into portable containers, making it easier to build, ship, and run applications across different environments. Docker Compose simplifies the deployment of multi-container applications by defining the dependencies and relationships between the services, enabling developers to break down their applications into smaller, modular components that can be tested and deployed independently. Docker and Docker Compose can be used in a variety of scenarios, from development and testing to staging and production. Volumes and networks are also important concepts in Docker, allowing developers to persist data between Docker container runs and to connect containers to different types of networks.

Spring Security is a powerful and highly customizable authentication and access-control framework for Java-based applications. It is a project under the Spring Framework and provides a comprehensive security solution for both web and non-web applications. With Spring Security, developers can easily secure their applications by defining security rules, implementing authentication and authorization mechanisms, and handling access-control decisions.

In this article, we will discuss the basics of Spring Security and how to set it up in a web application. We will also go over some of the key features and capabilities of the framework, such as authentication and authorization, and how to secure different types of resources in an application.

Setting up Spring Security in a Web Application

To set up Spring Security in a web application, we need to add the Spring Security dependencies to our project. The easiest way to do this is to use the Spring Initializer tool, which can generate a basic project structure with the necessary dependencies.

Once the dependencies are added, we need to configure Spring Security in our application. This can be done by creating a configuration class that extends the WebSecurityConfigurerAdapter class and overrides its methods.

Here is an example of a basic configuration class:

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    @Autowired
    private UserDetailsService userDetailsService;

    @Autowired
    private PasswordEncoder passwordEncoder;

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .authorizeRequests()
                .antMatchers("/").permitAll()
                .antMatchers("/admin/**").hasRole("ADMIN")
                .anyRequest().authenticated()
                .and()
            .formLogin()
                .loginPage("/login")
                .permitAll()
                .and()
            .logout()
                .permitAll();
    }

    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userDetailsService).passwordEncoder(passwordEncoder);
    }
}

In the above example, we are using the http object to define the security rules for our application. We are using the authorizeRequests() method to specify which resources in our application should be protected and which should be publicly accessible. In this case, we are allowing all requests to the root path (/) and only allowing requests from users with the "ADMIN" role to the /admin path. All other requests are required to be authenticated.

We are also configuring the form-based login and logout functionality using the formLogin() and logout() methods. In this case, we are using the default login page provided by Spring Security, but we can also specify a custom login page by providing a URL to the loginPage() method.

Finally, we are configuring the authentication manager to use our custom UserDetailsService and PasswordEncoder implementations.

Authentication and Authorization

One of the key features of Spring Security is its support for authentication and authorization. Authentication is the process of verifying a user's identity, while authorization is the process of determining what actions a user is allowed to perform.

Spring Security provides several built-in authentication mechanisms, such as form-based login, basic authentication, and token-based authentication. It also supports custom authentication providers, which allow developers to implement their authentication methods.

For example, in the configuration class above, we are using form-based login for authentication. However, we could also use basic authentication by configuring the http object as follows:

http
    .authorizeRequests()
        .anyRequest().authenticated()
        .and()
    .httpBasic();

In this case, the user will be prompted to enter their credentials using a basic authentication dialog provided by the browser.

Once a user is authenticated, Spring Security uses the user's roles and authorities to determine what actions they are allowed to perform. Roles and authorities are used to define access-control rules, and they can be assigned to users in a variety of ways, such as using the UserDetailsService interface or by reading them from a database or LDAP server.

For example, in the configuration class above, we are using the hasRole() method to restrict access to the /admin path to users with the "ADMIN" role. We could also use the hasAuthority() method to restrict access based on a specific authority, such as ROLE_ADMIN.

Securing Resources

Spring Security can be used to secure a wide variety of resources, including web pages, RESTful web services, and even individual methods in a Java class.

For example, in the configuration class above, we are using the authorizeRequests() method to define security rules for web pages. However, we can also use the antMatchers() method to define rules for specific web services or methods.

Here is an example of how to secure a RESTful web service:

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    // ...

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .authorizeRequests()
                .antMatchers("/").permitAll()
                .antMatchers("/admin/**").hasRole("ADMIN")
                .antMatchers(HttpMethod.POST, "/api/**").hasRole("USER")
                .anyRequest().authenticated()
                .and()
            // ...
    }
}

In this case, we are allowing all requests to the root path, allowing only users with the "ADMIN" role to access the /admin path, and allowing only users with the "USER" role to make POST requests to the /api path.

We can also use the @PreAuthorize and @PostAuthorize annotations to secure individual methods in a Java class. For example:

@Service
public class MyService {
    @PreAuthorize("hasRole('ADMIN')")
    public void doAdminTask() {
        // ...
    }

    @PostAuthorize("hasRole('USER')")
    public void doUserTask() {
        // ...
    }
}

In this case, the doAdminTask() method can only be accessed by users with the "ADMIN" role, and the doUserTask() method can only be accessed by users with the "USER" role.

Using Custom Roles in Spring Security

Spring Security supports the use of custom roles in addition to the built-in roles such as "ROLE_USER" and "ROLE_ADMIN". Custom roles can be used to define specific permissions and access controls for your application.

For example, let's say you have a music streaming application and you want to give premium users the ability to download songs. You can create a custom role called "ROLE_PREMIUM" and assign it to users who have a premium subscription. Then, you can use the hasRole() or hasAuthority() method to restrict access to the song download feature to users with the "ROLE_PREMIUM" role.

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    // ...

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .authorizeRequests()
                .antMatchers("/download/**").hasRole("PREMIUM")
                .anyRequest().permitAll()
                .and()
            // ...
    }
}

In this case, only users with the "ROLE_PREMIUM" role will be able to access the "/download" path.

Connecting Users to an External Database

In most real-world applications, user data is stored in a database such as MySQL, PostgreSQL, or MongoDB. Spring Security provides a UserDetailsService interface that can be used to load user data from an external database.

UserDetailsService is an interface with a single method, loadUserByUsername(String username), that takes a username as an argument and returns a UserDetails object. UserDetails is a Spring Security interface that represents a user and their roles and authorities.

Here is an example of how to implement a UserDetailsService that loads user data from a MySQL database:

@Service
public class MyUserDetailsService implements UserDetailsService {

    @Autowired
    private UserRepository userRepository;

    @Override
    public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {
        User user = userRepository.findByUsername(username);
        if (user == null) {
            throw new UsernameNotFoundException(username);
        }
        return new MyUserPrincipal(user);
    }
}

In this example, we are using a UserRepository to load user data from a MySQL database. The UserRepository is an interface that extends CrudRepository, which is a Spring Data interface for working with databases.

Once you have implemented a UserDetailsService, you need to configure Spring Security to use it. You can do this by overriding the configure(AuthenticationManagerBuilder auth) method in the configuration class, like so:

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    // ...

    @Autowired
    private MyUserDetailsService userDetailsService;

    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userDetailsService);
    }
}

With this configuration, Spring Security will use the MyUserDetailsService to load user data from the MySQL database.

It's important to note that when using an external database, you should also take care of security measures such as password encryption and securely storing the password. Spring Security provides built-in support for password encoding and provides several password encoders such as BCryptPasswordEncoder, SHA-256PasswordEncoder, etc.

Here is an example of how to configure Spring Security to use the BCryptPasswordEncoder:

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    // ...

    @Autowired
    private MyUserDetailsService userDetailsService;

    @Bean
    public PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth
            .userDetailsService(userDetailsService)
            .passwordEncoder(passwordEncoder());
    }
}

In this example, we have defined a passwordEncoder() bean that returns a new BCryptPasswordEncoder and we have passed this password encoder to the AuthenticationManagerBuilder. This will ensure that all passwords are encoded using the BCryptPasswordEncoder before being compared with the stored passwords.

It's also worth mentioning that when working with an external database, you should also handle user's roles and authorities. It can be done by creating tables in the database to store roles and authorities of each user and then mapping them to UserDetails object.

Managing Roles and Authorities in Spring Security with JPA and Hibernate

When working with an external database, it's important to handle user's roles and authorities. Instead of writing raw SQL queries, we can use the power of JPA and Hibernate to handle the database operations. In this section, we will see how to implement this with JPA and Hibernate.

First, we need to create the entities that will represent the tables in the database. These entities will be used to map the tables in the database to Java objects. Here are examples of the entities that will represent the users, authorities, roles, and user_roles tables:

@Entity
@Table(name = "users")
public class UserEntity {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;

    @Column(unique = true, nullable = false)
    private String username;

    @Column(nullable = false)
    private String password;

    @Column(nullable = false)
    private boolean enabled;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "user")
    private Set<AuthorityEntity> authorities;

    @ManyToMany(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
    @JoinTable(name = "user_roles", joinColumns = {
            @JoinColumn(name = "username", nullable = false, updatable = false) },
            inverseJoinColumns = { @JoinColumn(name = "role_id",
                    nullable = false, updatable = false) })
    private Set<RoleEntity> roles;
}

@Entity
@Table(name = "authorities")
public class AuthorityEntity {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;

    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "username", nullable = false)
    private UserEntity user;

    @Column(nullable = false)
    private String authority;
}

@Entity
@Table(name = "roles")
public class RoleEntity {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;

    @Column(unique = true, nullable = false)
    private String name;

    @ManyToMany(fetch = FetchType.LAZY, mappedBy = "roles")
    private Set<UserEntity> users;
}

Once the entities are created, we need to create a UserDetailsService implementation that loads the user data from the database and maps it to the UserDetails object. Here is an example of a UserDetailsService implementation that loads the user data from the database using JPA and Hibernate:

@Service
public class MyUserDetailsService implements UserDetailsService {

    @Autowired
    private UserRepository userRepository;

    @Override
    public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {
        UserEntity userEntity = userRepository.findByUsername(username);
        if(userEntity == null) {
            throw new UsernameNotFoundException("Invalid username or password.");
        }
        return new org.springframework.security.core.userdetails.User(userEntity.getUsername(), userEntity.getPassword(), getAuthority(userEntity));
    }

    private List<SimpleGrantedAuthority> getAuthority(UserEntity user) {
        return user.getRoles().stream().map(role -> new SimpleGrantedAuthority(role.getName())).collect(Collectors.toList());
    }
}

As you can see, we are using a UserRepository interface that extends the JpaRepository interface. This interface will handle all the database operations related to the UserEntity class. The findByUsername method is used to find a user by its username, and the getAuthority method is used to map the user's roles to a SimpleGrantedAuthority object.

Conclusion

In conclusion, Spring Security is a powerful framework that provides a lot of features out of the box. In this article, we have covered some of the most important features of Spring Security and how to use them in a real-world application. We have discussed how to secure web applications with Spring Security and how to use authentication and authorization in Spring Security. We have also covered how to use custom roles and authorities in Spring Security and how to connect the users to an external database using JPA and Hibernate. With the help of these examples and explanations, you should now have a good understanding of how to use Spring Security in your web applications.

This tutorial is part of Springing into Action: A Spring Boot Journey from Novice to Pro Series, be sure to check it out for more related content!

Spring provides several powerful features for unit testing, including the ability to easily create mock objects and integrate them into the application context. In this article, we will explore how to use Mockito, a popular mocking framework, in conjunction with Spring to write effective unit tests.

Setup

To get started, you will need to add the following dependency to your project's build file:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-starter-test</artifactId>
    <version>2.4.0</version>
    <scope>test</scope>
</dependency>

This spring-starter-test dependency includes a number of useful libraries for testing Spring-based applications, including JUnit, Hamcrest, and Mockito.

Introduction to Unit Testing

Unit testing is a software testing method in which individual units or components of a software application are tested in isolation from the rest of the system. The goal of unit testing is to validate that each unit of the software application is working as intended.

Unit tests are typically written by developers and are used to test the functionality of small, isolated components of the application such as methods or classes. Unit tests are automated and are run frequently during development to ensure that changes to the code do not break existing functionality.

Benefits of Unit Testing

• Early detection of bugs and errors in the code.

• Increased confidence in the code and the ability to make changes without fear of introducing new bugs.

• Improved code quality and design.

• Faster development times as developers can quickly and easily make changes to the code.

Unit testing is just one type of testing and it's not enough to ensure that the whole application is working as expected. There are other types of testing as well, such as:

• Integration testing: testing how different units or components of the application work together.

• Functional testing: testing the functionality of the application from the user's perspective.

• Acceptance testing: testing the application to ensure that it meets the requirements of the end user.

In general, unit testing is the foundation of software testing and provides a good starting point for more comprehensive testing. It's important to have a good unit testing strategy in place to ensure that the application is working as intended and to reduce the risk of bugs and errors.

Unit testing is an important part of software development and it's beneficial to have a good understanding of how to write and execute unit tests. Spring Boot provides a convenient way to write unit tests with its built-in support for testing, along with the tools such as JUnit, Mockito, and Spring Test.

Understanding Mockito Framework

Mockito is a popular open-source testing framework for Java that allows developers to create mocks and stubs for their unit tests. It is designed to make it easy to write tests for your code by providing a simple, clean, and intuitive API. One of the key features of Mockito is its ability to create mocks and stubs of objects, which are used to simulate the behavior of real objects in a test environment.

Mocks

A mock object is a simulation of a real object that is used in place of the real object in a test environment. In Mockito, a mock object can be created using the mock method or the @Mock annotation. The mock method creates a mock object of a specific class, while the @Mock annotation creates a mock object of a field that is annotated with the @Mock annotation.

For example, we can create a mock of the MyService class like this:

MyService myService = mock(MyService.class);

or

@Mock
MyService myService;

Spies

A spy object, on the other hand, is a special type of mock object that allows you to call the real methods of the object while still being able to specify the behavior of certain methods. In Mockito, a spy object can be created using the spy method or the @Spy annotation. The spy method creates a spy object of a specific class, while the @Spy annotation creates a spy object of a field that is annotated with the @Spy annotation.

For example, we can create a spy of the MyService class like this:

MyService myService = spy(new MyService());

or

@Spy
MyService myService = new MyService();

Stubs

A stub is an object that you can use to simulate the behavior of a real object in a test environment. In Mockito, a stub can be created using the when method. The when method is used to specify the behavior of a method for a specific input.

For example, we can use the when method to specify the behavior of the isValid method for a specific input:

when(myService.isValid("valid")).thenReturn(true);

Differences between @MockBean, @SpyBean and @Stub

@MockBean is a Spring annotation that creates a mock of a bean and adds it to the Spring context. This means that the mock object will be used in place of the real object when it is injected into other beans.

@SpyBean is also a Spring annotation that creates a spy of a bean and adds it to the Spring context. This means that the spy object will be used in place of the real object when it is injected into other beans, but it will still call the real methods of the object.

@Stub is not a Spring annotation it is a term used to describe the use of the when method from the Mockito framework, which is used to specify the behavior of a method for a specific input.

In general, @MockBean and @SpyBean are used to create mocks and spies of objects that are used in the Spring context, while the when method is used to create stubs of methods for specific inputs. It's important to note that the usage of these annotations and methods depend on the specific needs of your test case and the behavior you want to simulate.

For example, if you want to test the behavior of a service class that makes a call to a repository class, you might use a @MockBean annotation to create a mock of the repository class so that you can control its behavior in the test. On the other hand, if you want to test the behavior of a service class that makes a call to a external API, you might use the when method to create a stub of the API call and control its behavior in the test.

In summary, the key difference between using @MockBean, @SpyBean and when method is that @MockBean and @SpyBean are used to create mocks and spies of objects that are used in the Spring context, while the when method is used to create stubs of methods for specific inputs.

Configuration Class for SpringBootTest

In addition to the annotations and methods discussed above, Spring also provides a way to configure the test context through the use of configuration classes. These classes can be used to specify which beans should be loaded for the test and which should be excluded.

For example, the following configuration class can be used to only load the beans that are needed for the test and exclude the others:

@Configuration
@Import({ Service.class, Repository.class })
public class TestConfig {

}

This configuration class can then be used in the test class as follows:

@RunWith(SpringRunner.class)
@SpringBootTest(classes = TestConfig.class)
public class MyTest {
    ...
}

This way we can include only the classes that are needed for the test and exclude the others. This can be especially useful when we have a large application and we only want to test a specific part of it.

It is also possible to specify the configuration class while using @DataJpaTest, @WebMvcTest, @JsonTest and other annotations that provide a slice of the context.

@WebMvcTest(controllers = MyController.class, config = TestConfig.class)

This is particularly useful when you want to test only specific layers of your application, such as the web layer or the data access layer.

Spring Starter Test Usage

One of the most powerful features of Spring for unit testing is the ability to easily create mock objects and integrate them into the application context. We can use the @MockBean annotation to create a mock of a bean and automatically add it to the application context. For example:

@MockBean
private MyService myService;

In this example, a mock of the MyService bean is created and automatically added to the application context. We can then use this mock to write test cases and verify that the correct methods are called with the correct arguments.

We can also use the @SpyBean annotation to create a spy of a bean instead of a mock. Spies are useful when we want to test the behavior of a bean while still calling the real methods.

The @TestExecutionListeners annotation can be used to specify one or more test execution listeners that will be used for the test class. The DependencyInjectionTestExecutionListener is one of the built-in listeners provided by Spring and it can be used to control the order of test methods.

For example, we can use the @Dependencies annotation to specify the order in which methods should be executed:

@TestExecutionListeners(DependencyInjectionTestExecutionListener.class)
@Dependencies({ TestMethod1.class, TestMethod2.class })
public class Test {
    // test methods here

    @Test
    public void testMethod1() {
        // test code here
    }

    @Test
    public void testMethod2() {
        // test code here
    }
}

In the example above, we are using the @TestExecutionListeners annotation to specify that the DependencyInjectionTestExecutionListener will be used for the test class. This listener can be used to control the order of test methods, for example by specifying the @Dependencies annotation.

To test with the context re-usage, we can use the @DirtiesContext annotation. This annotation can be used to indicate that a test method has dirtied the context, and that the context should be rebuilt before the next test method is executed. For example:

@Test
@DirtiesContext
public void testMethod() {
    // test code here
}

In this example, the context will be rebuilt before the next test method is executed.

There are also ways to control the order of test classes and methods, for example using the @TestPropertySource annotation. This annotation can be used to specify properties that should be used for a test class or method. For example:

@TestPropertySource(properties = { "property1=value1", "property2=value2" })
public class Test {
    // test methods here
}

In this example, the properties property1 and property2 will be set to value1 and value2 respectively for all the test methods in this class.

Another way to control the order of test classes and methods is by using the @TestMethodOrder annotation. This annotation can be used to specify the order in which test methods should be executed. For example:

@TestMethodOrder(OrderAnnotation.class)
public class Test {
    // test methods here

    @Order(1)
    @Test
    public void testMethod1() {
        // test code here
    }

    @Order(2)
    @Test
    public void testMethod2() {
        // test code here
    }
}

In this example, the testMethod1 will be executed before the testMethod2 because it is annotated with @Order(1) and testMethod2 is annotated with @Order(2).

Code Examples

Here is an example of a simple unit test using Mockito and Spring:

@RunWith(SpringRunner.class)
@SpringBootTest
public class MyServiceTest {

    @MockBean
    private MyRepository repository;

    @Autowired
    private MyService service;

    @Test
    public void testGetData() {
        when(repository.findById(1L)).thenReturn(new MyData(1L, "Test Data"));
        MyData data = service.getData(1L);
        assertEquals(1L, data.getId());
        assertEquals("Test Data", data.getValue());
        verify(repository).findById(1L);
    }
}

In this example, we are using the @MockBean annotation to create a mock of the MyRepository bean and @Autowired annotation to inject the MyService bean into the test class. We are then using the when method from Mockito to specify the behavior of the mock repository and using the verify method to check that the findById method is called with the correct argument.

Here is an example of a unit test using @SpyBean and @DirtiesContext:

@RunWith(SpringRunner.class)
@SpringBootTest
public class MyServiceTest {

    @SpyBean
    private MyService service;

    @Test
    public void testDoSomething() {
        doReturn(true).when(service).isValid();
        service.doSomething();
        verify(service).isValid();
        verify(service).saveData();
    }

    @Test
    @DirtiesContext
    public void testDoSomethingElse() {
        doReturn(false).when(service).isValid();
        service.doSomething();
        verify(service, never()).saveData();
    }
}

In this example, we are using the @SpyBean annotation to create a spy of the MyService bean, which allows us to test the behavior of the bean while still calling the real methods. In the testDoSomething method, we are using the doReturn method from Mockito to specify the behavior of the isValid method and then verifying that the saveData method is called. In the testDoSomethingElse method, we are also using the doReturn method to specify the behavior of the isValid method, but this time we are expecting the saveData method to not be called. We are also using the @DirtiesContext annotation on this method to indicate that the context should be dirtied after this test method is executed. This means that any changes made to the context during this test method will not affect the other test methods.

Mocking Static Methods in Classes for Tests

In some cases, we may need to test a class that has static methods or a static class itself. Mocking static methods can be a bit tricky as mockito does not support mocking of static methods by default, but it can be done by using the Mockito.mock() with Mockito.spy() .

Here is an example of how to use Mockito.spy() to mock a static method:

public class MyTest {
    @Test
    public void testStaticMethod() {
        MyStaticClass spy = Mockito.spy(MyStaticClass.class);
        Mockito.doReturn("mocked result").when(spy).staticMethod();
        String result = spy.staticMethod();
        assertEquals("mocked result", result);
    }
}

In this example, we use the Mockito.spy() method to create a spy of the static class, and the Mockito.doReturn() method is used to specify the behavior of the static method.

It's worth noting that using Mockito.spy() is not recommended in some cases as it's not always recommended to test static methods and classes because in some cases it might be better to refactor the code and make it testable by using dependency injection and interfaces.

Parallel Execution of Tests

Parallel execution of tests is a way to speed up the testing process by running multiple tests at the same time. This can be achieved in Spring Boot by configuring the properties file and specifying the number of threads to use for running tests.

Parallel Execution within the Same Class

To run tests in parallel within the same class, we can use the @Test(threadPoolSize = X, invocationCount = Y) annotation. threadPoolSize represents the number of threads to use for running the tests, and invocationCount represents the number of times the test method should be invoked. For example, if we have a test class with 4 test methods and we want to run them in parallel using 2 threads, the configuration would look like this:

@Test(threadPoolSize = 2, invocationCount = 4)
public void testMethod() {
    // test logic here
}

Parallel Execution Across Classes

To run tests in parallel across different classes, we can use the @Test(threadPoolSize = X) annotation at the class level. This will run all the test methods within the class in parallel using the specified number of threads. For example, if we have 2 test classes with 4 test methods each and we want to run them in parallel using 2 threads, the configuration would look like this:

@Test(threadPoolSize = 2)
public class TestClass1 {
    @Test
    public void testMethod1() {
        // test logic here
    }
    @Test
    public void testMethod2() {
        // test logic here
    }
    // additional test methods
}

@Test(threadPoolSize = 2)
public class TestClass2 {
    @Test
    public void testMethod1() {
        // test logic here
    }
    @Test
    public void testMethod2() {
        // test logic here
    }
    // additional test methods
}

You can also use the spring.test.parallel.execution.enabled and spring.test.parallel.execution.strategy in the application.properties file to configure the parallel execution.

spring.test.parallel.execution.enabled=true
spring.test.parallel.execution.strategy=classes

The strategy property can take values of classes and methods. classes runs all the test classes in parallel and methods runs all the test methods within a class in parallel.

It's worth noting that parallel execution may cause some test methods to fail due to the shared state or unexpected order of test execution. Therefore, it's important to make sure that your test methods are independent and not dependent on the execution order.

Conclusion

In conclusion, Spring unit testing with Mockito is a powerful combination for testing Spring applications. We have covered various features and annotations such as @MockBean, @SpyBean, and @TestExecutionListeners that allow us to fine-tune our tests and ensure that they are accurate and reliable. Additionally, we discussed the differences between mock, spy, and stub annotations and the usage scenarios for each.

Moreover, we also covered the configuration class for SpringBootTest and how to specify only some classes to run for testing and not the whole context, this is useful when we want to test a specific set of classes and not all the classes in the application. Also, we discussed how to mock static methods in classes for tests and testing static classes with code examples.

Finally, we discussed the importance of unit testing and its benefits, including the different types of testing such as unit tests and integration tests. Additionally, we covered the parallel execution of tests in the same class or parallel execution of tests across classes using the properties file in SpringBootTests and the differences and configurations needed with examples in code and execution time.

In summary, Spring unit testing with Mockito is a powerful tool for ensuring the reliability and accuracy of your Spring applications. With the various features and annotations provided by Spring and Mockito, you can fine-tune your tests to suit your specific needs and ensure that your application is working as expected.

Spring Boot Actuator is a powerful tool for monitoring and managing your Spring Boot applications at runtime. It provides a wide range of information about the application and its environment; and allows for the management of the application. In this article, we will explore the features of Spring Boot Actuator and how to add it to your project, usage and code examples to help you understand how it can improve the reliability and maintainability of your Spring Boot applications.

What is Spring Boot Actuator?

Spring Boot Actuator is a sub-project of Spring Boot that provides advanced monitoring and management capabilities for Spring Boot applications. It provides a set of endpoints that allow you to monitor and manage your application at runtime. These endpoints provide information about the application, such as its health, metrics, and configuration, as well as allow you to perform operations on the application, such as refreshing its configuration or shutting it down.

Adding Spring Boot Actuator to Your Project

Adding Spring Boot Actuator to your project is easy. All you need to do is add the spring-boot-starter-actuator dependency on your project's pom.xml file.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Once you've added the dependency, you can start using Actuator's endpoints by visiting http://localhost:8080/actuator in your browser.

Available Endpoints

Spring Boot Actuator provides a wide range of endpoints that allow you to monitor and manage your application at runtime. Here are some of the most commonly used endpoints:

Health Endpoint

The health endpoint provides information about the health of your application. It returns a JSON object that contains the status of the application, as well as any additional information that may be useful for monitoring the application's health. By default, the health endpoint returns only the status of the application, but you can configure it to return additional information by implementing the HealthIndicator interface.

For example, you can create a custom HealthIndicator that returns the current time:

@Component
public class TimeHealthIndicator implements HealthIndicator {

    @Override
    public Health health() {
        String currentTime = LocalDateTime.now().toString();
        return Health.up().withDetail("time", currentTime).build();
    }
}

Metrics Endpoint

The metrics endpoint provides information about the performance of your application. It returns a JSON object that contains various metrics, such as the number of requests per second, the average response time, and the memory usage of the application. You can configure which metrics are exposed by Actuator by using the management.metrics.export property.

Info Endpoint

The info endpoint provides information about the application, such as its version and build time. You can configure the information that is exposed by Actuator by using the info property in the application.properties file. For example, you can add the following properties to the file:

info.app.name=My Application
info.app.version=1.0

Configuration Endpoint

The configprops endpoint provides information about the configuration of your application. It returns a JSON object that contains all the properties defined in the application.properties or application.yml file, as well as any other configuration properties that are defined by the application's dependencies. This can be useful for troubleshooting configuration issues and for understanding the configuration of your application at runtime.

Beans Endpoint

The beans endpoint provides information about the beans that are defined in your application's context. It returns a JSON object that contains all the beans that are defined, as well as information about their scope, type, and dependencies. This can be useful for understanding how your application is constructed and for troubleshooting issues related to bean initialization.

Threads Endpoint

The threaddump endpoint provides information about the threads that are currently running in your application. It returns a JSON object that contains information about each thread, such as its name, state, and call stack. This can be useful for troubleshooting performance issues and for understanding how your application is utilizing resources.

Shutdown Endpoint

The shutdown endpoint allows you to safely shut down your application. It is disabled by default, but you can enable it by setting the management.endpoint.shutdown.enabled property to true. Once it is enabled, you can send a POST request to the shutdown endpoint to shut down the application. This can be useful for performing maintenance tasks or for shutting down the application in response to an error.

Extending Actuator Endpoints

Actuator provides a wide range of endpoints out of the box, but you may find that you need to add your custom endpoints to your application. Spring Boot Actuator makes it easy to extend the existing endpoints or to define your custom endpoints.

Extending Existing Endpoints

You can extend existing endpoints by creating a new class that implements the appropriate Endpoint interface. For example, if you want to add additional information to the health endpoint, you can create a new class that implements the HealthEndpoint interface.

@Component
public class CustomHealthEndpoint implements HealthEndpoint {

    private final HealthEndpoint delegate;

    public CustomHealthEndpoint(HealthEndpoint delegate) {
        this.delegate = delegate;
    }

    @Override
    public Health health() {
        Health health = delegate.health();
        // Add custom information to the health object
        return health;
    }
}

You can also use @EndpointExtension to extend existing endpoints. This annotation is used to indicate that a bean should be used as an extension of an existing endpoint.

@Component
@EndpointExtension(endpoint = HealthEndpoint.class)
public class CustomHealthEndpointExtension {
    // Add custom information to the health object
}

Defining Custom Endpoints

You can define your custom endpoints by creating a new class that implements the Endpoint interface. For example, you can create a new endpoint that returns information about the current time:

@Component
@Endpoint(id = "time")
public class TimeEndpoint implements Endpoint<String> {

    @Override
    public String invoke() {
        return LocalDateTime.now().toString();
    }

    @Override
    public boolean isEnabled() {
        return true;
    }

    @Override
    public boolean isSensitive() {
        return false;
    }
}

You can also use @Endpoint to define custom endpoints. This annotation is used to indicate that a bean should be registered as a new endpoint.

@Component
@Endpoint(id = "custom")
public class CustomEndpoint {

    @ReadOperation
    public String customEndpoint() {
        return "Hello from custom endpoint";
    }
}

Once you have defined your custom endpoint, you can access it by sending a GET request to the /actuator/{id} endpoint, where {id} is the ID of your custom endpoint.

By extending and defining custom endpoints, you can add new functionality to your Actuator endpoints and customize them to suit the needs of your application.

Conclusion

Spring Boot Actuator is a powerful tool for monitoring and managing your Spring Boot applications at runtime. It provides a wide range of endpoints that allow you to monitor and manage your application, from its health and performance to its configuration and resources. Actuator also allows you to extend and define custom endpoints to suit the needs of your application. By adding Actuator to your project and understanding its capabilities, you can improve the reliability and maintainability of your Spring Boot applications.

Microservices architecture has become increasingly popular in recent years as a way to build scalable and resilient applications. However, building and maintaining a microservices-based system can be challenging, especially when it comes to service discovery, load balancing, and fault tolerance. Netflix OSS (Open Source Software) is a set of tools and frameworks that can help to address these challenges and make it easier to build and maintain a microservices-based system. In this article, we will take a look at how to integrate Netflix OSS with a Spring Boot application to improve its reliability, scalability, and performance.

Eureka

Eureka is a service discovery and registry tool. It allows services to register themselves and discover other services in the system. In a microservices-based system, services often need to call other services to perform their tasks. Eureka makes it easy for services to find and call other services by maintaining a registry of all the services in the system.

To integrate Eureka with our Spring Boot application, we first need to add the Spring Cloud Netflix Eureka dependency to our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

Next, we need to enable Eureka in our service by adding the @EnableEurekaClient annotation to our main class.

@SpringBootApplication
@EnableEurekaClient
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
}

We also need to configure the Eureka server URL in our application properties.

eureka.client.service-url.default-zone=http://localhost:8761/eureka/

With these configurations in place, our service will register itself with the Eureka server and be discoverable by other services in the system.

Ribbon

Ribbon is a client-side load balancer. It allows a service to distribute load among multiple instances of a dependent service. In a microservices-based system, it is common for a service to have multiple instances to handle increased traffic. Ribbon makes it easy for a service to distribute load among these instances, improving the system's scalability.

To integrate Ribbon with our Spring Boot application, we first need to add the Spring Cloud Netflix Ribbon dependency to our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>

Next, we need to use the RestTemplate class with @LoadBalanced annotation to make a call to dependent services.

@Autowired
@LoadBalanced
private RestTemplate restTemplate;

public void callDependentService() {
    restTemplate.getForObject("http://my-service/endpoint", String.class);
}

With these configurations in place, Ribbon will automatically handle load balancing when calling the "my-service" endpoint, distributing the load among the specified server instances.

Hystrix

Hystrix is a latency and fault tolerance library. It helps to prevent cascading failures in a microservices-based system by providing a fallback mechanism when a service fails. It also provides a circuit breaker pattern that can stop further calls to a service if it fails too many times, preventing the system from being overwhelmed by failed requests.

To integrate Hystrix with our Spring Boot application, we first need to add the Spring Cloud Netflix Hystrix dependency to our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>

Next, we need to use the @HystrixCommand annotation on the methods that call dependent services.

@HystrixCommand(fallbackMethod = "defaultFallbackMethod")
public String callDependentService() {
    return restTemplate.getForObject("http://my-service/endpoint", String.class);
}

public String defaultFallbackMethod() {
    return "Fallback method called.";
}

With these configurations in place, Hystrix will automatically handle fallback and circuit breaker functionality when calling the "my-service" endpoint, providing a fallback mechanism when the service fails and preventing the system from being overwhelmed by failed requests.

Zuul

Zuul is a routing and filtering tool. It acts as a reverse proxy, routing incoming requests to the appropriate service and filtering requests based on specified rules. In a microservices-based system, Zuul can be used to handle authentication, rate limiting, and other request-level functionality that would otherwise need to be implemented in each service.

To integrate Zuul with our Spring Boot application, we first need to add the Spring Cloud Netflix Zuul dependency to our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>

Next, we need to enable Zuul in our service by adding the @EnableZuulProxy annotation to our main class.

@SpringBootApplication
@EnableZuulProxy
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
}

We also need to configure the routing rules in our application properties.

zuul.routes.my-service.path=/my-service/**
zuul.routes.my-service.service-id=my-service

With these configurations in place, Zuul will handle routing incoming requests to the "my-service" endpoint to the appropriate service and filtering requests based on specified rules.

Archaius

Archaius is a configuration management tool. It allows for dynamic configuration of services and provides a consistent configuration across all instances of a service. In a microservices-based system, Archaius can be used to handle configuration changes without the need for a service restart.

To integrate Archaius with our Spring Boot application, we first need to add the Spring Cloud Netflix Archaius dependency to our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-archaius</artifactId>
</dependency>

Next, we need to enable Archaius in our service by adding the @EnableArchaius annotation to our main class.

@SpringBootApplication
@EnableArchaius
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
}

We also need to configure the Archaius properties in our application properties.

archaius.configurationSource.additionalUrls=file:///etc/config/application.properties

This configuration tells Archaius to read the additional configuration from the specified file location. Archaius also supports other sources of configuration like, AWS DynamoDB, Zookeeper, etc.

With these configurations in place, Archaius will handle dynamic configuration changes without the need for a service restart. This makes it easy to make changes to the configuration of a service without having to take it down for maintenance.

Governator

Governator is a library that provides enhanced lifecycle management for applications. It is built on top of Google Guice and provides features such as annotation-based configuration, lifecycle callbacks, and JMX integration.

To use Governator in our Spring Boot application, we first need to add the following dependency to our project:

<dependency>
    <groupId>com.netflix.governator</groupId>
    <artifactId>governator-core</artifactId>
    <version>latest version</version>
</dependency>

Next, we can use Governator's annotation-based configuration feature by annotating our configuration class with @ConfigurationProperties.

@ConfigurationProperties
public class MyConfiguration {
    private String property1;
    private int property2;
    //getters and setters
}

We can also use Governator's lifecycle callbacks by implementing the LifecycleListener interface and registering it in our main class.

public class MyLifecycleListener implements LifecycleListener {
    public void start() throws Exception {
        // do something on start
    }
    public void stop() throws Exception {
        // do something on stop
    }
}

@SpringBootApplication
public class MyApplication {
    public static void main(String[] args) {
        Injector injector = LifecycleInjector.builder().withModules(new MyModule()).build().createInjector();
        injector.getInstance(LifecycleManager.class).start();
    }
}

Governator also provides JMX integration by exposing the objects managed by Guice as MBeans, this way we can use JMX clients to monitor and manage our application.

Scoop

Scoop is a metrics aggregation tool. It can be used to collect, aggregate, and visualize metrics from multiple services in a microservices-based system. Scoop provides an HTTP endpoint that can be used to collect metrics from different services.

To use Scoop in our Spring Boot application, we first need to add the following dependency to our project:

<dependency>
    <groupId>com.netflix.scoop</groupId>
    <artifactId>scoop-core</artifactId>
    <version>latest version</version>
</dependency>

Next, we need to configure Scoop by adding the following properties to our application properties:

scoop.metrics.export.interval=10
scoop.metrics.export.endpoint=http://localhost:9090/metrics

This configuration tells Scoop to export metrics every 10 seconds to the specified endpoint. With this in place, our application will start exporting metrics to the specified endpoint, where they can be collected and aggregated by Scoop.

RxJava

RxJava is a Java implementation of the ReactiveX programming model. It can be used to build highly concurrent, fault-tolerant, and responsive systems.

To use RxJava in our Spring Boot application, we first need to add the following dependency to our project:

<dependency>
    <groupId>io.reactivex.rxjava2</groupId>
    <artifactId>rxjava</artifactId>
    <version>latest version</version>
</dependency>

Once we have the dependency, we can start using RxJava's Observable and Observer classes to create reactive streams of data. For example, we can use the Observable.create() method to create an observable that emits a sequence of integers:

Observable<Integer> observable = Observable.create(subscriber -> {
    for (int i = 0; i < 10; i++) {
        subscriber.onNext(i);
    }
    subscriber.onComplete();
});

We can then subscribe to this observable and consume the emitted items:

observable.subscribe(new Observer<Integer>() {
    public void onNext(Integer item) {
        System.out.println("Next: " + item);
    }
    public void onError(Throwable error) {
        System.err.println("Error: " + error.getMessage());
    }
    public void onComplete() {
        System.out.println("Complete");
    }
});

RxJava also provides several operators that can be used to transform, filter, and combine observables. For example, we can use the map() operator to transform the items emitted by an observable:

Observable<String> mappedObservable = observable.map(item -> "Number: " + item);

RxJava can be used to build highly concurrent, fault-tolerant, and responsive systems by providing an easy way to work with asynchronous data streams. It can be used in combination with other Netflix OSS tools like Hystrix to provide a complete solution for building resilient microservices.

Conclusion

Netflix OSS provides a rich set of tools and frameworks that can be used to build robust and scalable microservices-based systems. By integrating these tools into our Spring Boot application, we can leverage their capabilities to handle service discovery, load balancing, circuit breaker, routing, and dynamic configuration management. The tools and frameworks provided by Netflix OSS are battle-tested and widely used in production systems, making them a reliable choice for building microservices-based systems.

To further explore the usage of these tools and frameworks, I recommend checking out the official Netflix OSS documentation and the Spring Cloud Netflix documentation. Additionally, there are also many tutorials and examples available online that can help you get started with integrating Netflix OSS into your Spring Boot application.

Java Persistence API (JPA) is a Java specification for managing, persisting, and accessing data between Java objects/classes and a relational database. Hibernate is a popular implementation of the JPA specification and is widely used in the Java community. In this article, we will take a look at how to use JPA and Hibernate in Spring Boot applications with code examples and in-depth explanations.

Why use JPA and Hibernate in Spring Boot?

JPA and Hibernate are great technologies for persisting data in a Java application. They provide a standard way of mapping Java objects to a relational database and allow for easy management of data in the application. Additionally, using JPA and Hibernate in a Spring Boot application allows for easy integration with other Spring features such as transactions and caching. They provide a simple and efficient way to manage data in a relational database. In this article, we will talk about setting up JPA & Hibernate, creating Entities & Repositories and then we will dive into the more advanced features of JPA and Hibernate, specifically custom queries and Hibernate Query Language (HQL).

Setting up JPA and Hibernate in Spring Boot

Before we can start using JPA and Hibernate in our Spring Boot application, we need to set up a few dependencies in our pom.xml file. First, we need to add the Spring Data JPA dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

This dependency will bring in the necessary JPA and Hibernate libraries as well as the Spring Data JPA library, which provides easy integration with the JPA and Hibernate libraries.

Next, we need to add a database driver dependency. For example, if we are using MySQL, we would add the following dependency:

<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
</dependency>

Once we have these dependencies set up, we can start configuring our application to use JPA and Hibernate.

Configuring JPA and Hibernate

To configure JPA and Hibernate in our Spring Boot application, we need to create a application.properties file in the src/main/resources directory. In this file, we will specify our database connection information and other JPA and Hibernate settings.

For example, to configure a MySQL database, we would add the following properties to our application.properties file:

spring.datasource.url=jdbc:mysql://localhost:3306/dbname
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.hibernate.ddl-auto=update

The spring.datasource.url, spring.datasource.username, and spring.datasource.password properties are used to configure the connection to the MySQL database. The spring.jpa.hibernate.ddl-auto property is used to configure Hibernate to automatically update the database schema.

Creating Entities

Now that we have our application configured to use JPA and Hibernate, we can start creating entities. Entities are plain old Java objects (POJOs) that represent the data in our application. They are annotated with JPA annotations to indicate how they should be mapped to the database.

For example, let's say we have a simple application that stores information about books. We could create a Book entity like this:

@Entity
public class Book {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String title;

    private String author;

    private String ISBN;

    // Getters and setters
}

The @Entity annotation indicates that this class is an entity and should be mapped to a database table. The @Id annotation indicates that the id field is the primary key for the table. The @GeneratedValue annotation is used to configure the primary key to be generated automatically by the database.

NOTE: You can use Lombok's annotations in conjunction with the above @Entity annotation to generate all the boilerplate code; if you want to learn more about the topic, you can check out my article Spring Boot & Lombok annotations: A deeper look

Creating Repositories

Once we have our entities defined, we can start creating repositories. Repositories are used to interact with the data in the database. In Spring Data JPA, we can create a repository by creating an interface that extends the JpaRepository interface.

For example, to create a repository for our Book entity, we could create an interface like this:

public interface BookRepository extends JpaRepository<Book, Long> {
}

This interface will provide basic CRUD functionality for our Book entity and can be easily injected into our service layer.

Using Transactions

JPA and Hibernate also provide support for transactions. Transactions are used to group multiple database operations together and ensure that they are either all committed or all rolled back. In a Spring Boot application, we can use the @Transactional annotation to indicate that a method should be executed within a transaction.

For example, let's say we have a service method that creates a new book:

@Service
public class BookService {

    @Autowired
    private BookRepository bookRepository;

    @Transactional
    public Book createBook(String title, String author, String ISBN) {
        Book book = new Book();
        book.setTitle(title);
        book.setAuthor(author);
        book.setISBN(ISBN);
        return bookRepository.save(book);
    }
}

The @Transactional annotation on the createBook method indicates that the method should be executed within a transaction. This means that if an exception is thrown during the execution of the method, the database will roll back any changes made during the method.

Custom Queries

JPA provides a simple and convenient way to create custom queries using the @Query annotation. This annotation allows you to define a query directly in the repository interface, rather than in a separate class or XML file. The @Query annotation takes a single parameter, the query string. The query string can be written in either JPQL (Java Persistence Query Language) or SQL.

The following example shows how to use the @Query annotation to create a custom query that retrieves all users with a specific role:

@Repository
public interface UserRepository extends JpaRepository<User, Long> {

    @Query("SELECT u FROM User u WHERE u.role = ?1")
    List<User> findByRole(String role);
}

In this example, the findByRole method is annotated with the @Query annotation and the query string is written in JPQL. The ?1 parameter is a positional parameter that is replaced with the value of the role argument when the query is executed.

It's also possible to use named parameters instead of positional parameters. Named parameters are prefixed with a : and can be used in the query string by referencing them by name. The following example shows how to use named parameters:

@Repository
public interface UserRepository extends JpaRepository<User, Long> {

    @Query("SELECT u FROM User u WHERE u.role = :role")
    List<User> findByRole(@Param("role") String role);
}

It's important to note that the use of custom queries can result in more complex codebase and harder to maintain, that's why it is essential to use best practices and to write test cases to cover all the use cases.

Hibernate Query Language (HQL)

HQL is an object-oriented query language that is similar to SQL, but it operates on the objects of the persistent classes rather than on the database tables. HQL is used to create queries that are executed by the Hibernate framework.

The following example shows how to use HQL to create a query that retrieves all users with a specific role:

Session session = sessionFactory.openSession();
Query query = session.createQuery("FROM User WHERE role = :role");
query.setParameter("role", "admin");
List<User> users = query.list();

In this example, the createQuery method is used to create a new query. The query string is written in HQL and the :role parameter is a named parameter that is replaced with the value of the role variable when the query is executed.

Using HQL can be more powerful than using JPA's @Query annotation, as it allows you to execute more complex queries and to use advanced features of the Hibernate framework. However, it can also result in more complex codebase and harder to maintain, that's why it's important to use it judiciously and with caution.

Best Practices

When using custom queries and HQL in your SpringBoot application, there are a few best practices to keep in mind:

1. Use parameter binding: Instead of concatenating variables into the query string, use parameter binding to prevent SQL injection attacks.

2. Keep queries simple: Complex queries can be difficult to maintain and may result in poor performance. Try to keep queries as simple as possible.

3. Use pagination: When retrieving large amounts of data, use pagination to limit the number of results returned and to improve performance.

4. Test your queries: Always test your queries to ensure that they are returning the expected results and to check for any performance issues.

5. Logging: Enable the logging of SQL statements to monitor the performance and identify any issues

Conclusion

In this article, we delve into the advanced features of JPA and Hibernate in a Spring Boot application. We cover the setup and configuration of the necessary dependencies, as well as the creation of entities and repositories. Additionally, we explore the use of transactions and best practices for implementing custom queries and HQL. These features, when used correctly, can greatly improve the performance and efficiency of data retrieval operations in your application. It's important to keep in mind, however, that they should be used cautiously to maintain a maintainable codebase and optimal performance. By following the guidelines outlined in this article, you can harness the full potential of JPA and Hibernate in your Spring Boot application. As always, testing and monitoring your code is crucial to ensure optimal performance.

Java is a powerful and versatile programming language, and one of its most powerful features is its support for multithreading. Multithreading allows a program to run multiple threads in parallel, which can greatly improve the performance and responsiveness of a program. In this article, we will take a comprehensive look at threads in Java 8+ and see how they can be used to improve the performance of a program.

What are Threads?

A thread is a lightweight, independent unit of execution. It is a separate flow of control that runs in parallel with the main program. Each thread has its own stack, program counter, and local variables, which allows it to run independently of other threads. In Java, threads are implemented using the Thread class and the Runnable interface.

A Thread is a class in the Java standard library that represents a single thread of execution. A Thread object can be created and started using the start() method. A Runnable is an interface that defines a single method, run(), which is used to define the code that will be executed by the thread.

public class MyThread implements Runnable {
    public void run() {
        // Code to be executed by the thread
    }
}

Thread thread = new Thread(new MyThread());
thread.start();

Thread States

A thread can be in one of several states, including:

• NEW: The thread has been created but has not yet been started.

• RUNNABLE: The thread is running or is able to run.

• BLOCKED: The thread is blocked and is waiting for a lock or other resource.

• WAITING: The thread is waiting for another thread to perform a specific action.

• TIMED_WAITING: The thread is waiting for a specific amount of time.

• TERMINATED: The thread has completed execution.

A thread's state can be accessed using the getState() method of the Thread class.

Thread Priorities

In Java, threads can be assigned a priority, which determines the order in which they are executed. The Thread class defines several constants for the different priority levels, including MIN_PRIORITY, NORM_PRIORITY, and MAX_PRIORITY. The default priority for a new thread is NORM_PRIORITY.

Thread thread = new Thread(new MyThread());
thread.setPriority(Thread.MIN_PRIORITY);
thread.start();

It's important to note that setting a thread's priority does not guarantee that it will be executed before other threads with lower priorities. The actual order in which threads are executed depends on the underlying operating system and the JVM's thread scheduler.

Synchronization

Synchronization is a mechanism that allows multiple threads to access shared resources in a controlled manner. In Java, synchronization is achieved through the use of the synchronized keyword and the Lock and Condition interfaces.

Synchronized Methods

A synchronized method is a method that can only be executed by one thread at a time. To make a method synchronized, the synchronized keyword is used before the method's return type.

public class MyThread implements Runnable {
    public synchronized void run() {
// Code to be executed by the thread
}
}

When a thread enters a synchronized method, it acquires the lock for the object that the method belongs to. This means that no other thread can enter any synchronized method of that object until the first thread has exited the method and released the lock.

Synchronized Blocks

In addition to synchronized methods, Java also supports synchronized blocks. A synchronized block is a block of code that can only be executed by one thread at a time. To create a synchronized block, the synchronized keyword is used followed by the object that the block will synchronize on.

This can be useful when you only need to synchronize a specific section of code rather than an entire method. For example:

public class MyThread implements Runnable {
    public void run() {
        synchronized(this) {
            // Code to be executed by the thread
        }
    }
}

In the above example, we've created a synchronized block that synchronizes on the this object. This means that only one thread can execute the code inside the block at a time.

Volatile Variables

Another way to control the visibility of variables between threads is by using the volatile keyword. A variable that is declared as volatile will have its value immediately visible to all other threads. This can be useful when multiple threads need to read and write to the same variable.

public class MyThread implements Runnable {
    private volatile boolean running = true;
    public void run() {
        while(running) {
            // Code to be executed by the thread
        }
    }
    public void stop() {
        running = false;
    }
}

In the above example, we've created a volatile variable named running. This variable is used to control the execution of the run() method. When the stop() method is called, it sets the value of running to false, causing the thread to exit the while loop and end execution.

Thread Pools

Creating and managing threads can be a time-consuming and resource-intensive task. To help with this, Java provides a thread pooling mechanism through the Executor framework. A thread pool is a group of pre-initialized, reusable threads that can be used to execute multiple tasks simultaneously.

The Executor interface provides the execute() method for submitting tasks to be executed by a thread in the pool. The Executors class provides several methods for creating and managing thread pools, such as newFixedThreadPool() and newCachedThreadPool().

Executor executor = Executors.newFixedThreadPool(10);
executor.execute(new MyThread());

In the above example, we've created a fixed thread pool with a maximum of 10 threads. We then submit a task (an instance of MyThread) to be executed by a thread in the pool.

Multithreading Issues

As we have seen, threading is a powerful feature that can greatly improve the performance and responsiveness of your applications. However, multithreading can also introduce new challenges, such as race conditions, deadlocks, and thread safety, that must be handled with proper care. In this section, we will delve deeper into these topics and explore solutions, best practices, and code examples for handling these challenges.

Race Conditions

A race condition occurs when multiple threads access shared resources simultaneously and the outcome of the program depends on the order in which the threads execute. This can lead to unexpected and unpredictable behavior, such as data corruption or incorrect results.

One common solution to race conditions is to use synchronization to control access to shared resources. The synchronized keyword can be used to create a critical section of code that can only be executed by one thread at a time. This ensures that only one thread can access the shared resource at a time, eliminating the possibility of a race condition.

public class MyThread implements Runnable {
    private List<Integer> sharedList = new ArrayList<>();

    public void run() {
        synchronized(sharedList) {
            // Access shared resource
            sharedList.add(Thread.currentThread().getId());
        }
    }
}

In the above example, we've created a synchronized block that synchronizes on the sharedList object. This means that only one thread can execute the code inside the block at a time, ensuring that the sharedList is not accessed simultaneously by multiple threads.

Another solution to race conditions is the use of the Atomic classes provided by the java.util.concurrent.atomic package. These classes provide a way to perform atomic operations on variables, such as incrementing or decrementing a value, without the need for explicit synchronization.

AtomicInteger atomicInt = new AtomicInteger();

public void increment() {
    atomicInt.incrementAndGet();
}

In the above example, we've created an AtomicInteger and we can increment it's value using incrementAndGet method which is an atomic operation that is guaranteed to be thread-safe.

Deadlocks

A deadlock occurs when two or more threads are blocked, waiting for each other to release a resource. This results in the threads being unable to proceed, effectively locking the program.

One solution to deadlocks is to use a Lock object instead of the synchronized keyword. A Lock object provides more fine-grained control over thread synchronization and also provides a way to detect and recover from deadlocks.

public class MyThread implements Runnable {
    private Lock lock1 = new ReentrantLock();
    private Lock lock2 = new ReentrantLock();

    public void run() {
        lock1.lock();
        try {
            // Access shared resource
            lock2.lock();
            try {
                // Access shared resource
            } finally {
                lock2.unlock();
            }
        } finally {
            lock1.unlock();
        }
    }
}

In the above example, we've created two Lock objects and used them to control access to the shared resources. By using the try-finally pattern, we ensure that the locks are always released, even if an exception is thrown. This helps to prevent deadlocks caused by threads holding on to resources indefinitely.

Another solution to deadlocks is to use the Lock.tryLock() method, which attempts to acquire a lock and returns immediately with a boolean value indicating whether the lock was acquired or not. This can be used to implement a timeout mechanism for acquiring locks, which can help to avoid deadlocks caused by threads waiting for resources that will never be released.

public class MyThread implements Runnable {
    private Lock lock = new ReentrantLock();

    public void run() {
        if (lock.tryLock(1, TimeUnit.SECONDS)) {
            try {
                // Access shared resource
            } finally {
                lock.unlock();
            }
        } else {
            // Handle failure to acquire lock
        }
    }
}

In the above example, we've used the tryLock method to attempt to acquire the lock with a timeout of 1 second. If the lock is acquired, the thread can access the shared resource. If the lock is not acquired within the timeout period, the thread can handle the failure and take appropriate action, such as retrying later or giving up.

Thread-safety

Thread-safety is the property of an object or a class that can be safely accessed by multiple threads without causing unexpected behavior. To ensure thread-safety, we can use the same solutions as mentioned above: synchronization and atomic variables, as well as using thread-safe collections from java.util.concurrent package.

public class MyThreadSafeClass {
    private AtomicInteger atomicInt = new AtomicInteger();
    private ConcurrentHashMap<String, String> threadSafeMap = new ConcurrentHashMap<>();

    public void increment() {
        atomicInt.incrementAndGet();
    }

    public void put(String key, String value) {
        threadSafeMap.put(key, value);
    }
}

In the above example, we've used an AtomicInteger to ensure that the increment method is thread-safe and a ConcurrentHashMap to ensure that the put method is thread-safe.

Conclusion

Threading can greatly improve the performance and responsiveness of your applications, but it also introduces new challenges such as race conditions, deadlocks, and thread-safety that must be handled with proper care. In this article, we covered the basics of thread creation, synchronization, and thread pools in Java 8+ to create robust and efficient concurrent applications. Additionally, we explored solutions, best practices, and code examples for handling the challenges of multithreading such as using synchronization, atomic variables, and thread-safe collections. By understanding and applying these techniques, you can develop multithreaded applications that are both efficient and safe. I hope this article has provided you with a solid foundation for working with threads in your own Java applications.

Spring Boot and Lombok are two popular frameworks used in Java development. They both provide a set of annotations that can make your code more concise and readable. In this article, we will take a deeper look at some of the most commonly used annotations in Spring Boot and Lombok, their uses and how they can help you write better code.

Spring Boot Annotations

• @SpringBootApplication: This is a convenience annotation that is equivalent to including the @Configuration, @EnableAutoConfiguration, and @ComponentScan annotations. It is used to enable auto-configuration and component scanning in your application.

    @SpringBootApplication public class Application {     
        public static void main(String[] args) { 
            SpringApplication.run(Application.class, args);     
        } 
    }
  

• @RestController: This annotation is used to create a RESTful controller in Spring Boot. It is a combination of the @Controller and @ResponseBody annotations.

    @RestController
    public class MyController {
        @GetMapping("/")
        public String home() {
            return "Hello, World!";
        }
    }
  

• @RestControllerAdvice: This annotation is used to handle the global exception handling in the application.

    @RestControllerAdvice
    public class ExceptionAdvice {
        @ExceptionHandler(value = Exception.class)
        public ResponseEntity<Object> handleException(Exception ex) {
            return new ResponseEntity<>(ex.getMessage(), HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
  

• @Autowired: This annotation is used to inject a dependency into a class. It can be used on fields, methods, or constructors.

    @Service
    public class MyService {
        @Autowired
        private MyRepository myRepository;
    }
  

• @Value: This annotation is used to assign a value to a field from a properties file.

    @Value("${app.name}")
    private String appName;
  

• @PostConstruct: This annotation is used to indicate a method to be executed after the bean is created and all the dependencies are injected.

    @Service
    public class MyService {
        @Autowired
        private MyRepository myRepository;
  
        @PostConstruct
        public void init() {
            //some initialization code
        }
    }
  

Lombok Annotations

• @Data: This annotation generates getters and setters, toString, equals, and hashCode methods for a class.

    @Data
    public class Student {
        private Long id;
        private String name;
    }
  

• @NoArgsConstructor: This annotation generates a constructor with no arguments.

    @NoArgsConstructor
    public class Student {
        private Long id;
        private String name;
    }
  

• @AllArgsConstructor: This annotation generates a constructor with all fields as arguments.

    @AllArgsConstructor
    public class Student {
        private Long id;
        private String name;
    }
  

• @Builder : This annotation generates a builder pattern for a class

    @Builder
    public class Student {
        private Long id;
        private String name;
    }
  

• @Slf4j: This annotation is used to generate a logger for the class

    @Slf4j
    public class MyService {
        public void myMethod() {
            log.info("This is an example of log");
        }
    }
  

These are just a few examples of the annotations provided by Spring Boot and Lombok. Using these annotations can help you write cleaner and more readable code, making it easier to maintain and understand.

In conclusion, Spring Boot and Lombok are powerful tools that can help you to write better code and improve your productivity as a developer. With the help of annotations, you can make your code more readable and maintainable. I hope this article has been helpful in introducing some of the most useful annotations provided by these frameworks.

In this tutorial, we will learn how to create a simple application using Spring Boot and JPA Repository, including pagination. We will use a MySQL database to persist our data and will create a REST API to perform CRUD operations on it.

Prerequisites

• Java 8 or later

• Maven 3.x

• MySQL Server

• Spring Boot CLI

• An IDE like Eclipse or IntelliJ IDEA

Setting up the Project

First, we need to create a new Spring Boot project using the Spring Initializer. We will select the following dependencies:

• Spring Web

• Spring Data JPA

• MySQL Driver

Once the project is generated, we can import it into our IDE and start adding our code.

Configuring the Database

Next, we will configure our MySQL database by adding the following properties to our application.properties file:

spring.datasource.url=jdbc:mysql://<host>:<port>/<dbname>
spring.datasource.username=<username>
spring.datasource.password=<password>

Make sure to replace the placeholders with the appropriate values for your MySQL server.

Creating the Entity

We will create two entities, Student and Course, to represent our data. The Student entity will have a one-to-many relationship with the Course entity.

@Entity
public class Student {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String name;
    @OneToMany(mappedBy = "student", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
    private List<Course> courses;
    // getters and setters
}

@Entity
public class Course {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String name;
    @ManyToOne
    @JoinColumn(name = "student_id")
    private Student student;
    // getters and setters
}

Creating the Repository

We will create a JPA repository for each entity to perform CRUD operations on our data.

public interface StudentRepository extends JpaRepository<Student, Long> {
}

public interface CourseRepository extends JpaRepository<Course, Long> {
}

Creating the REST API

We will create a REST controller for each entity to handle the HTTP requests.

@RestController
@RequestMapping("/students")
public class StudentController {
    @Autowired
    private StudentRepository studentRepository;
    @Autowired
    private CourseRepository courseRepository;

    @GetMapping
    public Page<Student> getAllStudents(Pageable pageable) {
        return studentRepository.findAll(pageable);
    }

    @PostMapping
    public Student addStudent(@RequestBody Student student) {
        return studentRepository.save(student);
    }

    @PutMapping("/{studentId}")
    public Student updateStudent(@PathVariable Long studentId, @RequestBodyStudent student) {
 student.setId(studentId); return studentRepository.save(student); }

@DeleteMapping("/{studentId}")
public void deleteStudent(@PathVariable Long studentId) {
    studentRepository.deleteById(studentId);
}

@GetMapping("/{studentId}/courses")
public List<Course> getStudentCourses(@PathVariable Long studentId) {
    return courseRepository.findByStudentId(studentId);
}
}

@RestController
@RequestMapping("/courses")
public class CourseController {
    @Autowired
    private CourseRepository courseRepository;

    @GetMapping
    public Page<Course> getAllCourses(Pageable pageable) {
        return courseRepository.findAll(pageable);
    }

    @PostMapping
    public Course addCourse(@RequestBody Course course) {
        return courseRepository.save(course);
    }

    @PutMapping("/{courseId}")
    public Course updateCourse(@PathVariable Long courseId, @RequestBody Course course) {
        course.setId(courseId);
        return courseRepository.save(course);
    }

    @DeleteMapping("/{courseId}")
    public void deleteCourse(@PathVariable Long courseId) {
        courseRepository.deleteById(courseId);
    }
}

In the above code examples, we have added pagination by passing a Pageable object to our repository's findAll() method. This allows us to retrieve a specific page of data from our database.

Running the Application

Once the project is set up, we can run it using the following command:

mvn spring-boot:run

We can now test our REST API by sending HTTP requests to the appropriate endpoints.

In this tutorial, we have learned how to create a simple application using Spring Boot and JPA Repository, including pagination. We have also seen how easy it is to perform CRUD operations on a MySQL database using this framework.

I hope you found this tutorial helpful, and that you can use this as a starting point for your next project.

Note: above code is a simplified version for demonstration purposes, it's not ready to use code, you may need to customize it according to your requirement.

Dependency Injection (DI) is a fundamental concept in software development that allows objects to be decoupled from their dependencies. In Spring, DI is achieved through the use of the @Autowired annotation, which can be applied to fields, methods, or constructors. DI is a powerful technique that can help you to write more maintainable and testable code.

However, when not used correctly, DI can lead to circular references. A circular reference occurs when two or more beans depend on each other, creating a cycle that cannot be resolved. In Spring, circular references can be detected and resolved through the use of @Lazy, @Primary, or @DependsOn annotations.

In addition to DI and circular references, understanding the Spring Bean Life Cycle is also important for developing robust and maintainable applications. The Spring Bean Life Cycle consists of several phases, including instantiation, dependency injection, initialization, and destruction. Understanding these phases and how to use annotations such as @PostConstruct and @PreDestroy can help you to write more efficient and effective code.

In this article, we will explore these topics in greater depth, providing examples and best practices for each.

Dependency Injection

Dependency Injection is a design pattern that allows objects to be decoupled from their dependencies. In Spring, this is achieved through the use of the @Autowired annotation.

The @Autowired annotation can be applied to fields, methods, or constructors, and is used to inject a dependency into a class. For example, if we have a MyService class that depends on a MyRepository class, we can use the @Autowired annotation to inject the MyRepository dependency into the MyService class:

@Service
public class MyService {
    @Autowired
    private MyRepository myRepository;

    public void doSomething() {
        // use myRepository here
    }
}

In the above example, the MyService class is annotated with @Service which tells Spring to create an instance of this class as a bean, and MyRepository class is also annotated with @Repository which tells Spring to create an instance of this class as a bean and the MyRepository bean is injected into the MyService bean.

We can also use the @Autowired annotation on constructors and methods. For example, instead of using the annotation on a field, we can use it on a constructor or method like this:

@Service
public class MyService {
    private final MyRepository myRepository;

    @Autowired
    public MyService(MyRepository myRepository) {
        this.myRepository = myRepository;
    }
    public void doSomething() {
        // use myRepository here
    }
}

In this example, Spring will automatically call the constructor with the MyRepository bean as its argument, and the myRepository field will be initialized with the injected dependency.

It's also worth noting that Spring also provides other annotations for injecting dependencies, such as @Resource, @Inject, and @Qualifier. These annotations can be used in similar ways as @Autowired, but they have some small differences. @Resource and @Inject are part of Java's standard annotation library, while @Autowired is specific to the Spring framework. @Qualifier can be used in conjunction with @Autowired to disambiguate between multiple beans of the same type.

@Service
public class MyService {
    @Autowired
    @Qualifier("myRepository")
    private MyRepository myRepository;
}

In the above example, @Qualifier is used to specify that the myRepository bean should be injected, rather than any other bean of type MyRepository.

Best Practices

• Use constructor injection where possible, as it makes the code more readable and testable.

• Use field injection only as a last resort, as it can make the code harder to understand and test.

• Avoid using setter injection, as it can lead to tight coupling between classes.

• Use @Qualifier to disambiguate between multiple beans of the same type.

• Use @Primary or @Lazy to resolve circular references.

Circular References

A circular reference occurs when two or more beans depend on each other, creating a cycle that cannot be resolved. In Spring, circular references can be detected and resolved through the use of @Lazy, @Primary, or @DependsOn annotations.

The @Lazy annotation can be used to delay the initialization of a bean until it is needed. For example, if we have a MyService class that depends on a MyRepository class, and the MyRepository class depends on the MyService class, we can use the @Lazy annotation on one of the dependencies to break the cycle:

@Service
public class MyService {
    @Autowired
    @Lazy
    private MyRepository myRepository;
}

@Repository
public class MyRepository {
    @Autowired
    private MyService myService;
}

In this example, the MyService bean will not be initialized until it is actually needed, thus breaking the circular reference.

The @Primary annotation can be used to specify that a particular bean should be used by default when multiple beans of the same type are found. For example, if we have two MyRepository beans, one annotated with @Primary and one without, the bean annotated with @Primary will be used by default.

@Repository
@Primary
public class MyRepositoryImpl1 implements MyRepository {
    //implementation
}

@Repository
public class MyRepositoryImpl2 implements MyRepository {
    //implementation
}

In this example, MyRepositoryImpl1 will be used by default as it is annotated with @Primary.

The @DependsOn annotation can be used to specify that a particular bean should be initialized before another bean. For example, if we have a MyService class that depends on a MyRepository class and a MyInitializer class, we can use the @DependsOn annotation to ensure that the MyInitializer bean is initialized before the MyRepository bean:

@Service
@DependsOn("myInitializer")
public class MyService {
    @Autowired
    private MyRepository myRepository;
}

@Component
public class MyInitializer {
    //initialization code
}

In this example, the MyInitializer bean will be initialized before the MyService bean, ensuring that the MyRepository bean has access to any resources that may have been initialized by MyInitializer.

Best Practices

• Use @Lazy to delay the initialization of a bean until it is actually needed.

• Use @Primary to specify that a particular bean should be used by default when multiple beans of the same type are found.

• Use @DependsOn to specify that a particular bean should be initialized before another bean.

• Avoid creating circular references whenever possible.

Spring Bean Life Cycle

The Spring Bean Life Cycle consists of several phases, including instantiation, dependency injection, initialization, and destruction. Understanding these phases and how to use annotations such as @PostConstruct and @PreDestroy can help you to write more efficient and effective code.

The @PostConstruct annotation can be used to indicate a method that should be called after the bean is constructed and all its dependencies have been injected. For example, if we have a MyService class that needs to perform some initialization after it has been constructed, we can use the @PostConstruct annotation on a method to achieve this:

@Service
public class MyService {
    @Autowired
    private MyRepository myRepository;

    @PostConstruct
    public void init() {
        //initialization code
    }
}

In this example, the init() method will be called after the MyService bean has been constructed and all its dependencies have been injected.

The @PreDestroy annotation can be used to indicate a method that should be called before the bean is destroyed. For example, if we have a MyService class that needs to perform some cleanup before it is destroyed, we can use the @PreDestroy annotation on a method to achieve this:

@Service
public class MyService {
    @Autowired
    private MyRepository myRepository;

    @PreDestroy
    public void cleanup() {
        //cleanup code
    }
}

In this example, the cleanup() method will be called before the MyService bean is destroyed.

It's also worth noting that Spring also provides other annotations for controlling the bean life cycle, such as @Bean and @Scope. The @Bean annotation is used to define a bean, while the @Scope annotation is used to define the scope of a bean (e.g. singleton, prototype, etc.).

Best Practices

• Use @PostConstruct to indicate a method that should be called after the bean is constructed and all its dependencies have been injected.

• Use @PreDestroy to indicate a method that should be called before the bean is destroyed.

• Use @Bean and @Scope to control the bean life cycle.

In conclusion, Dependency Injection, Circular References, and Spring Bean Life Cycle are all important concepts to understand when developing applications with Spring. By understanding these concepts and using the appropriate annotations, you can write more maintainable and testable code. Use these examples and best practices as a guide, and you'll be well on your way to mastering these topics.

The above article is an extensive guide on Dependency Injection, Circular References, and Spring Bean Life Cycle. It's a great resource for developers who want to learn more about these topics or want to improve their skills. The code examples in this article are in Java, but the concepts can be applied to other programming languages as well.

In this article, we will explore the benefits of using Spring Boot, a widely-used Java framework, for building web applications. Spring Boot, which is built on top of the Spring framework, offers a range of convenient tools and features that allow for fast and efficient development. We will also delve into how to get started with using Spring Boot for your next project.

What is Spring Boot?

Spring Boot is a framework for building web applications in Java. It is built on top of the Spring framework and provides a set of convenient tools and features for developing web applications quickly and efficiently. Some of the key features of Spring Boot include:

• Auto-configuration: Spring Boot automatically configures your application based on the dependencies you have included in your project. This means that you don't have to spend time manually configuring your application and can focus on writing your code.

• Embedded web servers: Spring Boot includes an embedded web server, so you don't have to set up and configure a separate web server. This makes it easy to run your application locally and deploy it to a production environment.

• Command-line interface: Spring Boot includes a command-line interface (CLI) that makes it easy to create new projects and run your application.

Getting Started

To get started with Spring Boot, you will need to have the following tools installed on your machine:

• Java Development Kit (JDK) version 8 or later

• Apache Maven

Once you have the necessary tools installed, you can create a new Spring Boot project using the Spring Initializer. The Spring Initializer is a web-based tool that helps you create new Spring Boot projects. To use it, simply go to the Spring Initializer website and select the options for your project.

Once you have created your project, you can open it in your favorite IDE and start developing your application.

Creating a Simple REST API

Now that you have a basic Spring Boot project set up, let's create a simple REST API. First, we need to add the spring-web dependency to our pom.xml file:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

Next, we will create a new class called HelloController:

@RestController
public class HelloController {

    @GetMapping("/hello")
    public String hello() {
        return "Hello, Spring Boot!";
    }
}

This class is a simple REST controller that maps the /hello endpoint to the hello() method. When this endpoint is called, the method will return the string "Hello, Spring Boot!".

Now, we can start our application and test our endpoint using a tool like curl or Postman:

curl http://localhost:8080/hello

You should see the following output:

Hello, Spring Boot!

Why Use Spring Boot?

Now that we've seen how easy it is to get started with Spring Boot and create a simple REST API, you may be wondering why you should use it for your next project. Here are a few reasons why:

• Productivity: Spring Boot makes it easy to create new projects and get started quickly. Its auto-configuration feature means that you don't have to spend time manually configuring your application and can focus on writing your code. This can significantly increase your productivity and help you get your application up and running faster.

• Scalability: Spring Boot is built on top of the Spring framework, which is a well-established and widely-used framework for building Java applications. This means that it is highly scalable and can handle large and complex projects.

• Flexibility: Spring Boot is highly configurable and allows you to choose the components and dependencies that you need for your project. This means that you can easily customize your application to meet your specific requirements.

• Community: Spring Boot is an open-source framework with a large and active community. This means that there are many resources available to help you learn and use the framework, as well as a wide range of third-party libraries and modules that you can use to extend the functionality of your application.

In conclusion, Spring Boot is an excellent choice for building web applications in Java. Its easy-to-use tools and features make it easy to get started and increase productivity, while its scalability, flexibility, and community support make it a great choice for large and complex projects. If you're looking for a framework to build your next web application, consider giving Spring Boot a try.

What Is Python?

Python is one of the most widely used programming languages. Guido van Rossum created it, and it was released in 1991.

Python is a dynamically semantic, interpreted, object-oriented high-level programming language. Its high-level built-in data structures, together with dynamic typing and dynamic binding, making it ideal for Rapid Application Development and as a scripting or glue language for connecting existing components.

Python's concise, easy-to-learn syntax prioritizes readability, which lowers software maintenance costs. Modules and packages are supported by Python, which fosters program modularity and code reuse. The Python interpreter and its substantial standard library are free to download and distribute in source or binary form for all major platforms.

Why is Python so well-liked?

Python is widely used for a variety of purposes. Here's a closer look at what makes it so flexible and user-friendly for programmers.

• It features a straightforward syntax that resembles normal English, making it easier to read and comprehend. This speeds up the development of projects as well as the improvement of existing ones.

• It's adaptable. Python can be used for a variety of purposes, including web development and machine learning.

• It's user-friendly, making it popular among new programmers.

• It's free to use and distribute, even for commercial purposes, because it's open source.

• The Python module and library archive—bundles of code developed by third-party users to extend Python's capabilities—is large and growing.

• Python has a vibrant community that contributes to the library of modules and libraries and serves as a resource for other programmers. Because of the large support network, finding a solution to a stumbling block is extremely simple; someone has almost certainly encountered the same issue before.

Install Python

There are various ways to install python, you can try to download it as a stand-alone from Python's Official Website or the most common way to install it with other Data science tools is through Anaconda

Anaconda is a Python and R programming language distribution aimed for simplifying package management and deployment in scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics, and so on).

After you install Python you can run the following command to check that it was installed successfully:

python --version

Most Common IDEs

There are a lot of IDEs which we can use to write Python programs, but the most commonly used ones - Not in a particular order- are as follows:

• PyCharm

• Visual Studio Code

• Jupyter Notebook

• PyDev

• Tabnine

• Atom

*Note: Jupyter comes pre-installed with Anaconda *

Getting Started With Python

Python is an interpreted programming language, which means that you write Python (.py) files in a text editor before passing them to the Python interpreter to run.

On the command line, you can run a python script like follows:

C:\Users\your name>python helloworld.py

The name of your python file should be "helloworld.py."

Let's begin by writing our first Python file, helloworld.py, in any text editor or IDE:

print("Hello, From Amr's Tech Universe Blog!")

The output should read:

Hello, From Amr's Tech Universe Blog!

Indentation

Instead of curly braces, Python utilises indentation for blocks. Both tabs and spaces are supported, however normal Python code must utilize four spaces to be indented properly.

Variables & Types

Python is an object-oriented programming language that is not "statically typed." You don't need to define variables or their types before utilizing them. In Python, every variable is an object.

Commenting

Comments are used to explain our code for others reading our code or for us so we don't forget what is done here

You can comment codes like this:

# This is a single line comment

"""
This is a multiline comment
in Python
"""

Variables

Variables are storage containers for data values. You can't declare a variable in Python like in strongly-typed languages; a variable is only created when you initially assign a value to a it

For example lets say that you want to define a variabl;e and assign the number 1997 to it, you can do it like this: myVar = 1997

now lets say that you changed your mind and want to store it as a string or a float even, In other languages you would have to define a new variable and cast the existing one to match the new variable's data type, but in Python you can do it as follows:

myVar = 1997

# Casting myVar to float:
myVar = float(myVar)
print(myVar) # 1997.0

# Returning myVar to int:
myVar = int(myVar) 
print(myVar) # 1997

# Casting myVar to string:
myvar = str(myVar)
print(myVar) # "1997"

Moreover, you can un-pack variable from a list, assign the same value to multiple variables and much more as we will show in the code example below:

# Multiple assignments
a, b, c = "Intro", "To", 'Python'

print(a)   # "Intro"
print(b)   # "To"
print(c)   # "Python"

# one value to multiple variables assignment
a, b, c = "Intro To Python" # or 'Intro To Python'

print(a)   # "Intro To Python"
print(b)   # "Intro To Python"
print(c)   # "Intro To Python"

# Unpacking lists
groupOfWords = ["Intro", "To", "Python"]
x, y, z = groupOfWords
print(x)   # "Intro"
print(y)   # "To"
print(z)   # "Python"

As you have seen, the variables in python are dynamic and change their type based on the value assigned to them as variables are treated like objects.

However, there are a few aspects to consider:

• Python is case-sensitive, so MyVar is different from myVar

• The name of a variable must begin with a letter or the underscore character

• A number cannot be the first character in a variable name

• String can be created with either " or ', the only difference is with " you can include apostrophes

• The type() method can be used to return the data type of a variable

Built-in-Data Types

Data type is a crucial notion in programming. Variables can store a variety of data, and different types can perform different tasks. Python comes with the following data types pre-installed in these categories:

• Text Type: str
• Numeric Types: int, float, complex
• Sequence Types: list, tuple, range
• Mapping Type: dict
• Set Types: set, frozenset
• Boolean Type: bool
• Binary Types: bytes, bytearray, memoryview

Lists

As we have shown in the previous code example, arrays and lists are extremely similar. They can contain any sort of variable, as well as an unlimited number of variables. It is also possible to iterate across lists in a very simple method.

Lists are ordered, which means that any newly added items are inserted at the end by default. They can hold multiple data types at the same time and also allow duplicates.

Lets show some examples to better understand what was just saiud and explore some of Lists functionalities:

myList = ["Amr", 25.0, 1997]

# Accessing items in list
name= myList[0]    # "Amr"
year= myList[-1] # Or myList[2] or myList[len(names) - 1]

print(myList)  #  ["Amr", 25.0, 1997]

# Add new items
myList.append("Hello") # Inserts at the end of List
myList.Insert(0, "New Item") # Inserts at the First position of the List

# Modify Items
myList[1] = "Second Item"

print(myList) #  ["New Item" ,"Second Item", 25.0, 1997]

# Delete Items
myList.pop() # Removes the last item in List
myList.remove("Second Item")

print(myList) #  ["New Item" , 25.0]

# List Looping
for item in myList:
    print(item) # {0} -> "New Item", {1} -> 25.0

for i in range(len(myList)):
    print(myList[i]) # {0} -> "New Item", {1} -> 25.0

List Comprehension

Whenever you want to create a new list based on the values of an existing list, list comprehension has a shorter syntax.

Taking the list from the past example and applying list comprehension on it:

"""
Instead of looping on all items by our selves to print them,
we can do something like this:
"""
[print(item) for item in myList] # {0} -> "New Item", {1} -> 25.0

Tuples

Tuples are a type of variable that allows you to store several elements in a single variable. A tuple is a collection of items that is both ordered and immutable.

Round brackets are used to write tuples. For example like GPS coordinates

egyptCoordinates = ("26.8206° N", "30.8025° E")
print(egyptCoordinates)  # ('26.8206° N', '30.8025° E')

# Access Items
print(egyptCoordinates[0])  # '26.8206° N'

We stated that tuples are immutable, but there is a workaround that can be done. By casting the tuple to a list, it becomes mutable again and we can update its items then create a new tuple with the updated values.

myTuple = ("Intro", "To", "Python")
myList = list(myTuple)
myList[1] = 0.0
myList.append(5)
myTuple = tuple(myList)

print(myTuple)  # ('Intro', 0.0, 'Python', 5)

If we want to assign all tuple values to variables as we did with lists, we can do as below:

# If number of variables is the same as items
fruits = ("apple", "banana", "cherry")

(green, yellow, red) = fruits

# If number of variables doesn't match the items
fruits = ("apple", "banana", "cherry", "strawberry", "raspberry")

(green, yellow, *red) = fruits

Note the use of * before the variable red, It is used to assign the excess items to the latest variable in the form of a list.

Dictionaries

Dictionaries are used to store data values in key:value pairs, they are mutable and don't allow duplicates in keys.

We will explore how to create, modify & delete dictionaries in the code example below:

person = {
  "name": "Amr Khaled",
  "gender": "Male",
  "age": 25.0,
  "occupation":{
   "company": "CIT VeriCash",
   "position": "Software Engineer"}
}

# Access Values
print(person["occupation"])  # {'company': 'CIT VeriCash', 'position': 'Software Engineer'}

The process of adding or modifying dictionaries is practically the same. If a key exists then it will be added, but if it is found then its value is updated

# Adding a new Key-Value Pair
person["hasCar"] = True

print(person) # {'name': 'Amr Khaled', 'gender': 'Male', 'age': 25.0, 'occupation': {'company': 'CIT VeriCash', 'position': 'Software Engineer'}, 'hasCar': True}

# Modifying a new Key-Value Pair
person["hasCar"] = False

print(person) # {'name': 'Amr Khaled', 'gender': 'Male', 'age': 25.0, 'occupation': {'company': 'CIT VeriCash', 'position': 'Software Engineer'}, 'hasCar': False}

Loops & Conditionals

Loops come in a variety of forms, and they're used to iterate over data and apply logic before moving on to the next item or returning a specific item.

They are often used in conjunction with logical conditionals and if ... else statements, that we will discuss shortly

For Loops

# Loop over a string
for x in "banana":
  print(x)   # {0} -> 'b', {1} -> 'a', ......, {5} ->'a'

# Loop over a list
for x in ["Intro", "To", "Python"]
  print(x)  # {0} -> 'Intro', {1} -> 'To', {2} -> 'Python'

# Loop over a range
for x in range(20):
  print(x)  #  {0} -> 0, {1} -> 1, ......, {19} -> 19

# Using Break statement
fruits = ["apple", "banana", "cherry"]
for x in fruits:
  if x == "banana":
    break # used to stop the for loop and resume the code execution

# Using Continue Statement
fruits = ["apple", "banana", "cherry"]
for x in fruits:
  if x == "banana":
    continue  # Skip the rest of the code in this iteration of loop and start with the next index

While Loops

We can use the while loop to run a series of statements as long as a condition is true.

# Print all numbers less than x
x = 100
i = 0
while i < x:
  print(i)
  i += 1
else:
  print("i = x")

If Statements

Assume that we are at a party and we are responsible for handling door access, we have to set some ground rules as to who will we allow to our party, maybe we want people to be above 21 or if they are on the invitation list, then we can allow them to enter, but how do we translate that to python code? Lets see, shall we?

minimumAge = 21
invitedPeopleList = ["Amr", "Maryam", "Anas", "Habiba", "Merna", "Nada", "Ayman", "May"]

person = {"name": "Ahmed", "age": 19}

isAllowedEntry = False # Initially everyone is denied entry

if person["name"] in invitedPeopleList or person["age"] >= minimumAge:
    isAllowedEntry = True
else:
    isAllowedEntry = False

*Note: The else statement above is redundant but was included for clarification purposes *

Functions

A function is a piece of code that only runs when it is invoked. It accepts data in the form of parameters and as a result of its operation, a function can return data in the form of one variable or more.

If we want to create a function that takes two integers as input and outputs a bunch of arithmetic operations - sum, multiplication, difference, reminder & power -

We can group all of those operations into a function and implement it as follows:

def getArithmeticOperations(x, y):
    sum = x+y
    diff = x-y
    multi = x*y
    power=x**y
    rem = x%y

    return sum, diff, multi, power, rem

a = 10
b = 2

sum, diff, multi, power, rem = getArithmeticOperations(a,b)

print(sum)  # 12
print(diff)   # 8
print(multi)  # 20
print(power)  # 100
print(rem)   # 0

Wow! that was a lot. If you are still feeling fuzzy or don't know how to memorize all of this new information, it is perfectly okay.

Remember that Rome, was not built in a day and that practice makes perfect. Try and familiarize yourself with more python syntax and problems and you will surely be on your way!

In this tutorial, I tried to give you the basics that I wished someone had told me about when I started my coding journey, this is just an introduction so you and I still have a long way to go together. But we all have to start somewhere.

Thank you so much for reading, hope you enjoyed it as much as I enjoyed writing it! See you soon with more blogs & tutorials in our Data Analysis, AI, Machine Learning and Everything In Between Series

Then you have come to the right place! In this blog you will be introduced to practical Data Analysis using SQL, and we will get to dive deeper into SQL and continue our previous SQL tutorial, If you haven't already then please go check How I Met Your Database -& Mainly Relational Using SQL-

Northwind Database

In this blog, we will be working with Northwind Database, which is a sample database created by Microsoft to practice fictional data in a company environment.

The database consists of the following tables:

• Suppliers: Suppliers and vendors of Northwind

• Customers: Customers who buy products from Northwind

• Employees: Employee details of Northwind traders

• Products: Product information

• Shippers: The names and contact information for the shippers who transport the goods from the traders to the end-users.

• Orders and Order_Details: Customers' and the companies' sales order interactions.

To have a broader idea of the database structure, we can check the below Entity Relationship Diagram -ERD-:

*Image credits goes to: yugabyte *

The Power Of SQL Queries

You now have a database that is completely ready for you to explore, so let's start by utilizing SQL and asking ourselves, what insights can we gain from our Database ?

Highest Performing Employees

Let's say that we measure our performance goals solely on the volume of orders, regardless of their monetary value.

We can use our SQL knowledge to know who the highest performing employees based on number of orders.

Based on the ERD above, the reference that links - Foreign Key - between the Orders & the Employees Tables is the Employee ID.

Foreign Keys

A FOREIGN KEY is a field (or set of fields) in one table that points to a PRIMARY KEY in another table. The child table is the table with the foreign key, and the referred or parent table is the table with the primary key.

SQL Query

The Query to get the volume of orders and the employee responsible looks like this:

SELECT E.EmployeeID As Employee_ID, 
COUNT(O.EmployeeID) AS Order_Count, 
CONCAT(E.FirstName, ' ' ,E.LastName) As Full_Name
FROM Orders as O
inner join Employees as E on O.EmployeeID = E.EmployeeID
group by E.EmployeeID, E.FirstName, E.LastName;

Lets dissect the query and understand what each part does, starting with GROUP BY

GROUP BY

The GROUP BY statement is used to groups rows with the same values into summary rows.

To group the result set by one or more columns, the GROUP BY statement is frequently used with aggregate functions (COUNT(), MAX(), MIN(), SUM(), AVG()).

In the above query however we can see that we grouped by The Employee's ID, First Name & Last Name.

Aliases

SQL aliases are used to give a table, or a column in a table, a temporary name, they are often used to make column names more readable.

An alias only exists for the duration of that query and is created with the AS keyword.

Inner Join

The INNER JOIN keyword picks records in both tables that have the same value. You can think about Inner Join in the term of the intersection between two tables

If the columns in both tables match, the INNER JOIN keyword selects all rows from both tables. These orders will not be displayed if there are records in the "Orders" table that do not have matches in the "Employees" table!

There are a lot of table join types, we will discuss them as the need arises but you can check the below Venn Diagram as it will provide you with a brief explanation for the use-case of each join type:

Query Output

Our executed query will yield the following result -as shown below- and this provided us with insight on which employees created the most orders for the customers.

Employee_ID    Order_Count     Full_Name
    1              123        Nancy Davolio
    2               96        Andrew Fuller
    3              127        Janet Leverling
    4              156        Margaret Peacock
    5               42        Steven Buchanan
    6               67        Michael Suyama
    7               72        Robert King
    8              104        Laura Callahan
    9               43        Anne Dodsworth

But this is still missing something fundamental, did you notice it?

If your answer was their order then you are absolutely correct!, our query didn't order the elements based on volume of orders, it is still missing the ORDER BY clause.

ORDER BY

As the name suggests, if we want to sort the result set in ascending or descending order, use the ORDER BY keyword.

By default, the ORDER BY keyword organizes the records in ascending order. Use the DESC keyword to sort the records in descending order.

For example, In our query, we should add the order by clause after the group by like this:

ORDER BY COUNT(O.EmployeeID) DESC;

OR

-- Since we defined Order_Count as an Alias for COUNT(O.EmployeeID)
-- we can use them interchangeably
order by Order_Count desc;

And that yeilds the result set organized as below:

Employee_ID    Order_Count     Full_Name
    4              156        Margaret Peacock
    3              127        Janet Leverling
    1              123        Nancy Davolio
    8              104        Laura Callahan
    2               96        Andrew Fuller
    7               72        Robert King
    6               67        Michael Suyama
    9               43        Anne Dodsworth
    5               42        Steven Buchanan

How have discounts affected the number of items sold?

It is a known fact that we all LOVE sales, no matter what the sale is on, we lways manage to find something that we like and buy something new or an even larger quantity of something that we use on a regular basis.

But, this is not a scientific way to prove it, so how can we ?

We can start by getting the number of items sold with a sale and those sold without while comparing them with the number of orders made.

Let's see how we can achieve that with SQL.

SQL Queries

We will be keeping the Queries simple and using only the basic syntax, but I strongly advice you to start reading about Left Join and joins in general and to start experimenting by yourself on various databases and datasets like The Famous Titanic Dataset on Kaggle

The below queries check for the total number of items purchased and the number of orders made.

-- For Items With Discounts:

SELECT SUM(OD.Quantity) AS Total_Items_Sold, Count(OD.OrderID)
FROM [Order Details] as OD
WHERE OD.Discount > 0;



-- For Items Without Discounts:

SELECT SUM(OD.Quantity) AS Total_Items_Sold, Count(OD.OrderID)
FROM [Order Details] as OD
WHERE OD.Discount = 0;

For the Results:

• Items With Discounts: A total of 22718 items were sold in 838 orders.

  Total_Items_Sold    Number_Of_Orders
        22718               838
  

• Items Without Discounts: A total of 28599 items in 1317 orders

  Total_Items_Sold    Number_Of_Orders
        28599               1317
  

If we calculate the average number of items per order, We will find that clearly the number of items purchased increases when a sale is present.

What discount percentage should the company offer?

Now that we have seen that discounts affect the volume of our sales, we have to determine the best percentage that will help us maximize our profit & market share.

Let's check the increments of 5% discounts in our database and see what our data is trying to tell us.

SQL QUERY

SELECT SUM(OD.Quantity) AS Total_Items_Sold, Count(OD.OrderID)AS Number_Of_Orders, OD.Discount AS Discount_Percentage
FROM [Order Details] as OD
WHERE OD.Discount = 0.05 or OD.Discount =0.1 or OD.Discount =0.2 or OD.Discount =0.25
GROUP BY OD.Discount
ORDER BY OD.Discount DESC;

This shows the following results, which surprisingly show a very little change in the order sizes and numbers is an indicator for the Northwind company that if the offer a discount as low as 5% , they will notice an increase in sales with no need to decrease their revenue any further.

Total_Items_Sold    Number_Of_Orders    Discount_Percentage
      4349                154                  0.25
      4351                161                  0.2
      4366                173                  0.1
      5182                185                  0.05

We can take this even a step further and see the effect of discounts on different product categories or If discounts affect certain regions more than others, but I will leave that for you to answer and test your skills.

This brings us to the end of today's blog about using SQL for Data Analysis, In the upcoming blog we will be getting to know Python and later on we will create a mini project with full code examples to handle the Northwind Database and other various datasets.

Hope you enjoyed reading this blog as much as I enjoyed writing it. If you have any comments or suggestions, please feel free to reach out ❤

However, things have altered considerably in recent years. Cloud computing has transformed and improved the way businesses plan, create, and sustain their IT infrastructure.

One of the most important aspects of this paradigm is Infrastructure as Code and that's what we'll explore today.

What Is Infrastructure as Code?

Infrastructure as Code (IaC) refers to the process of managing and supplying infrastructure using code rather than manual methods. It is a descriptive approach for managing infrastructure (networks, virtual machines, load balancers, and connection topologies) that employs the same versioning as the DevOps team does for source code.

An IaC model generates the same environment every time it is applied, similar to the principle that the same source code generates the same binary; with configuration files containing your infrastructure specifications created with IaC, making it easier to change and distribute configurations.

Why use IaC?

Now that the "what" is out of the way, lets focus on why do we want to utilize Iac and what is its competitive advantage.

Infrastructure administration and configuration were formerly done manually. Each environment has its own unique configuration that was set up by hand, and that resulted in a number of issues, including:

• Cost: You'll need to pay a lot of people to manage and maintain the infrastructure.

• Scaling: It takes time since it requires manual configuration of infrastructure operations, and that makes it difficult to handle surges in demand.

• Inconsistency: Human infrastructure configuration is prone to errors, there is inconsistency. Errors are unavoidable when multiple individuals configure different environments.

Whereas through IaC, you can achieve the following:

• Speed & Simplicity: IaC delivers the full infrastructure with the click of a button by just running a script.

• Consistency: All of it is specified as a code, which results in less errors than manual configuration.

• Risk minimization: It enables a larger group of individuals to collaborate more effectively on infrastructure configuration and administration.

• Cost Minimization: It will allow you to spend less time on routine infrastructure deployment activities and more time on more valuable tasks.

• Reusability: It improves reusability with minimal adjustments.

• Process Automation: As part of the continuous delivery process, it enables for the automation of the entire process from setup to removal.

Furthermore, describing infrastructure as code entails the following:

1. Allows infrastructure to be readily linked into version control techniques, making infrastructure modifications trackable and auditable.

2. Provides the ability to automate infrastructure management to a large extent. As a result of all of this, IaC is being integrated into CI/CD pipelines as a key component of the Software Development Life Cycle (SDLC).

3. Manual infrastructure provisioning and administration is no longer necessary. As a result, users can simply manage the underlying infrastructure's and configurations' unavoidable config drift while keeping all environments inside the intended configuration.

Challenges of IaC

Each new approach introduces a new set of issues, and IaC is no exception. The same characteristics that make IaC so effective and efficient also provide enterprises with some unique problems. Here's a quick rundown of what they're all about:

Accidental Destruction

In theory, once automated systems are up and running, they don't require ongoing supervision beyond periodic maintenance and replacement. In truth, even automated systems have issues, and these issues can build up over time to become major system-level disasters. It's also known as erosion in technical terms.

Configuration Drift

Automated configuration frequently results in infrastructure parts wandering over time. A repair applied to one server, for example, may not be reproduced across all servers. Although differences aren't necessarily negative, they must be documented and managed.

Lack of Expertise

Creating definition files and testing them to verify that they perform properly necessitates a thorough understanding of the organization's IT architecture. That's a unique set of abilities.

Lack of Proper Design & Planning

Many unknowns surround automation projects, so it's critical to identify and address them early on in the planning process. Continuous testing and phased execution of automation projects can provide sceptics with the information and confidence they need to lead more automation projects in the future.

Error Replication

It's simple to trace human actions and duplicate errors in manual operations. Error replication becomes a difficult problem when automation is involved. System administrators may not be able to correctly replicate error scenarios by analyzing log files, workflows, and other data.

Principles of IaC

1. Easy System Reproducibility

Your IaC approach should make it simple and quick to create and rebuild any part of your IT infrastructure. They shouldn't necessitate a lot of human work or complicated decision-making.

All of the tasks involved must be coded into the definition files, from selecting the software through configuring it. The scripts and tools that handle resource provisioning should have enough data to complete their duties without the need for human intervention.

2. Idempotence

Automated systems and their ability to handle difficult tasks are naturally viewed with suspicion by meticulous corporate leaders. As a result, no matter how many times IaC is run, it must maintain consistency. When new servers are installed, for example, they must be equal (or nearly identical) in capacity, performance, and dependability to the existing servers. This way, anytime new infrastructure parts are added, all decisions are automated and preset, from configuration to hosting names.

*Photo Credit: https://shahadarsh.com/ *

3. Repeatable Processes

System administrators have a natural predilection towards jobs that are easy to understand. When it comes to resource allocation, they like to do it the natural way: assess resource requirements, define best practices, and allocate resources.

Despite its effectiveness, such a procedure is anti-automation. IaC forces system administrators to think in terms of scripts. Their responsibilities must be divided down or grouped into repeatable processes that may be scripted.

4. Disposable Systems

To render hardware reliability insignificant to system operations, IaC relies heavily on reliable and robust software. Organizations must allow hardware failures to impact their businesses in the cloud era, since the underlying hardware may not always be trustworthy. As a result, software-level resource provisioning guarantees that in the event of a hardware breakdown, an alternate hardware allocation is quickly assigned to ensure that IT operations are not disrupted.

At the heart of IaC is fluid infrastructure that can be produced, destroyed, resized, transferred, and replaced. It should be able to seamlessly handle infrastructure changes such as resizing and expansions.

5. Ever-evolving Design

The IT infrastructure design is always changing to meet the organization's changing needs. Because infrastructure modifications are costly, businesses aim to keep them to a minimum by rigorously anticipating future requirements and designing systems accordingly. Future adjustments will be considerably more difficult and costly as a result of these unnecessarily complex designs.

This challenge is addressed by IaC-driven cloud infrastructure, which simplifies change management. While present systems are designed to fulfil current needs, future modifications must be simple to adopt. The best way to ensure that change management is simple and quick is to make regular changes so that all stakeholders are aware of the usual concerns and can develop scripts to successfully address them.

6. Self-Documentation

The teams are constantly striving to ensure their paperwork stay relevant, useful, and accurate. Someone may develop a detailed paper for a new procedure, but it is uncommon for such documents to be maintained up to date as changes and adjustments are made to the way things are done. In addition, holes in records occur on a frequent basis. Several people come up with their own shortcuts and tweaks. Some people build their scripts to make the process go more smoothly.

Despite the fact that documentation is frequently employed to maintain continuity, conventions, and even legal enforcement, it is an exaggerated portrayal of what occurs in reality. The scripts, definition files, and resources that implement the policy with infrastructure as code encapsulate the stages for carrying out a process. Only a small amount of additional documentation is required to get people started. The current documentation should be kept near to the idea that it records so that it is readily available when individuals make adjustments.

GitOps & IoC

GitOps is an operational framework that integrates DevOps best practices for application development to infrastructure automation, such as version control, collaboration, compliance, and CI/CD tooling. While the software development lifecycle has become increasingly automated, infrastructure continues to be a primarily human operation requiring specialized teams.

With the increasing demands on today's infrastructure, infrastructure automation has become increasingly important. Modern infrastructure must be able to handle cloud resources effectively in order to support continuous deployments.

Another approach to implement IaC is to use GitOps, which extends IaC and provides a procedure (Pull Request Process) for applying changes to Production or any other environment. It could also feature a control loop that checks that the actual state of the infrastructure matches the desired state on a routine basis. It will, for example, ensure that any modifications made directly to infrastructure revert to the desired state as defined by source control.

A real-life Example

A key restriction in software development is that the environment in which newly produced software code is tested must exactly match the actual environment in which such code will be deployed. The only method to ensure that new code does not conflict with current code definitions is to generate errors or conflicts that could endanger the entire system.

In the past, software delivery followed a pattern like this

A System Administrator would build up a physical server and install the operating system, including all necessary service packs and tuning, to match the status of the production environment's main running live machine.

The support database would then be handled by a Database Administrator, and the system would be handed over to a testing team.

The code/program would be delivered to the test machine by the developer, and the test team would execute many operational and compliance tests.

You can deploy the updated code to the live, operational environment once it has completed the full procedure. In many circumstances, the new code will not function properly, necessitating more troubleshooting and rework.

A live environment clone made with the same IaC as the live environment guarantees that if it works in the cloned environment, it will also work in the live environment.

Consider a software delivery pipeline that includes DEV, UAT, and Production environments. Given that those early environments are crucial for testing the quality and production readiness of a software build version, there appears to be no utility in having a DEV and UAT environment that isn't an exact mirror of the prod environment.

Virtualization made it possible to speed up this process, particularly when it came to constructing and maintaining a test server that would mimic the live environment. However, because the procedure was manual, a human would be required to develop and update the machine on a regular basis.

These processes got even more "AGILE" after the introduction of DevOps. Human intervention is replaced by automation in the server virtualization and testing phases, increasing productivity and efficiency.

It is now attainable for the developer to do all duties by himself

The developer creates the application code and configuration management-related instructions that will cause the virtualized environment, as well as other environments like the database, appliances, testing tools, delivery tools, and others, to take action.

The configuration management instructions will create a new virtual test environment with an application server and database instance that exactly mirrors the live operational environment structure, both in terms of service packs and versioning, as well as live data that is transferred to such virtual test environment, upon the delivery of new code. (This is the part of the process where Infrastructure as Code is used.)

Then, using a set of tools, the essential compliance checks, as well as error detection and resolution, will be carried out. After that, the updated code is ready to be deployed in a live IT environment.

Technologies & Tools

AWS CloudFormation, Red Hat Ansible, Chef, Puppet, SnowFlake, and Terraform are examples of infrastructure-as-code tools. Some tools employ a domain-specific language (DSL), while others use YAML or JSON as a standard template format.

When choosing a tool, companies should think about where they want to deploy it. AWS CloudFormation, for example, is designed to provision and manage infrastructure on AWS and integrates with other AWS services. Chef, on the other hand, works with on-premises servers as well as a variety of cloud provider infrastructure-as-a-service options.

Hope you enjoyed reading this blog as much as I enjoyed writing it. If you have any comments or suggestions, please feel free to reach out ❤

In this blog we will give an introduction on SQL, the basic syntax, the joins and everything you need to start your way in handling Data from relational databases in general.

Different types of Databases

We deal with databases on an almost daily basis, even without computers. Every time we access our bank accounts, add a friend on any social media application or buy anything online really.

In 1970, Edgar F. Codd of IBM published an academic paper titled, A Relational Model of Data for Large Shared Banks. The paper introduced an innovative way to model data. It paved the way of building a group of cross-linked tables that would allow you to store data just once. Introducing the capability to find answers to any data that we need so long as the answer was stored somewhere in it.

In the 1980s and ’90s, relational databases skyrocketed; Making the ability to query and index data extremely efficient. Table joins and Transactions were introduced.

There are various types of databases, that include but are not limited to:

Object-Oriented Databases

The sort of database that stores data in the database system using an object-based data paradigm. The information is represented and saved as objects, which are analogous to the objects used in object-oriented programming.

Object databases are frequently employed in applications that call for high performance, calculations, and quick results. Real-time systems, architectural & engineering for 3D modelling, telecommunications, and scientific products, molecular science, and astronomy are some of the common applications that use object databases.

Network Databases

The database is normally the one that follows the network data model. The data is represented in this manner as a network of nodes connected by links. It allows each record to have several children and parent nodes, forming a flexible network structure.

Cloud Databases

A database that stores information in a virtual environment and runs on a cloud computing platform. It offers users access to the database through a variety of cloud - based services. There are numerous cloud platforms available, however the below are the best:

• Amazon Web Services(AWS)
• Microsoft Azure
• Google Cloud SQL

NoSQL Databases

Non-SQL databases are a form of database that can store a wide range of data sets. It is not a relational database because it stores data in a variety of formats, not only tabular. It was created in response to a rise in the need for modern applications. As a result, in response to the demands, NoSQL introduced a wide range of database systems. A NoSQL database can be further classified into the following four types:

• Key-value storage: It is the most basic sort of database storage, in which each object is stored as a key (or attribute name) that holds its value.

• Document-oriented Database: A database that stores data in the form of a JSON-like document. It facilitates data storage for developers by using the same document-model format as the application code.

• Graph Databases: It's a graph-like structure for storing large volumes of data. The graph database is most typically used by social networking websites.

• Wide-column stores: It's akin to how data is stored in relational databases. Instead of storing data in rows, data is stored in huge columns.

Relational Databases

This database uses the relational data model, which stores data in the form of rows (tuples) and columns (attributes), which are combined to make a table (relation). SQL is used to store, manipulate, and maintain data in a relational database. Each table in the database has a key that distinguishes the data from that of other tables.

We will focus mainly in this blog on Relational databases and how to manage them using SQL.

What is SQL?

SQL stands for Structured Query Language, It is the way by which you can manipulate the database and perform various operations like Inserting, updating or deleting records and creating tables.

SQL became an ANSI standard -American National Standards Institute- in 1986, and of ISO -International Organization for Standardization- in 1987. However, there are different versions of the SQL language, they all support the major commands (such as SELECT, UPDATE, DELETE, INSERT, WHERE). You don't have to worry if all of that sounds gibberish to you right now as we will discuss each of these terms and more in details.

Relational Databases Properties

The ACID properties are the four most well-known properties of a relational model, and they are as follows:

1. A means Atomicity: This assures that the data operation will complete successfully or unsuccessfully. It employs an all-or-nothing approach. A transaction, for example, will either be committed or aborted.

2. C means Consistency: If we conduct any operation on the data, its value should be retained both before and after the operation. For example, the account balance should be correct before and after the transaction, i.e., it should be conserved.

3. I means Isolation: There can be multiple concurrent users accessing data from the database at the same time. As a result, data isolation should be maintained. When many transactions occur at the same time, for example, one transaction's effects should not be apparent to the database's other transactions.

4. D means Durability: It ensures that data modifications are permanent after the process is completed and the data is committed.

Most common SQL Commands

You will work with these commands in an almost daily basis, as they are essential in working with any query:

• CREATE DATABASE: creates a new database
• ALTER DATABASE: modifies a database
• CREATE TABLE: creates a new table
• ALTER TABLE: modifies a table
• DROP TABLE : deletes a table
• SELECT: extracts data from a database
• UPDATE: updates data in a database
• DELETE: deletes data from a database
• INSERT INTO: inserts new data into a database

Note: SQL keywords are not case sensitive: select is the same as SELECT and can be used interchangeably.

SQL Syntax

One or more tables are commonly seen in a database. A name is assigned to each table (e.g. "Customers" or "Purchases"). Tables include data records (rows).

We'll utilize our HIMYM knowledge and create a really simple database in this blog. The following is the script used to create the database, table and populating the table with some sample data:

IF NOT EXISTS(SELECT * FROM sys.databases WHERE name = 'HIMYM')
  BEGIN
    CREATE DATABASE HIMYM;
  END;

use HIMYM;

create table Characters(
    id INTEGER PRIMARY KEY IDENTITY (1, 1),
    CharacterName NVARCHAR(32) NOT NULL,
    Age FLOAT,
    Occupancy NVARCHAR(32)
);

INSERT INTO Characters(CharacterName,Age,Occupancy)
VALUES
('Ted Mosby', 27, 'Architect'),
('Marshall Eriksen', 27, 'Law-Student'),
('Barney Stinson', 29, 'PLEASE'),
('Lily Aldrin', 27, 'Pre-School Teacher'),
('Robin Scherbatsky', 25, 'News Reporter');

We performed a lot of operations in the script above , so let's provide a brief step-by-step explanation:

1. Checking if the database exists and if it does not, then we create it.
2. We then utilize the USE keyword to specify the database that we will use in the rest of the script to avoid any ambiguity.
3. Creating our Characters table with the specified columns (id, Character Name, Age & Occupation)
4. Populating the Characters table with some sample data.

We will dive deeper into PRIMARY KEYS, FOREIGN KEYS, IDENTITY & more SQL features and keywords in further blogs; so you do not have to worry about them right now if you do not understand them.

if we run the query SELECT * FROM Characters, we will retrieve the following data:

id    CharacterName         Age         Occupancy
1    Ted Mosby              27          Architect
2    Marshall Eriksen       27         Law-Student
3    Barney Stinson         29           PLEASE
4    Lily Aldrin            27      Pre-School Teacher
5    Robin Scherbatsky      25        News Reporter

We have five records, each row corresponds to a unique character with each column holding all information linked with a certain field.

Let's breakdown this simple select query structure and understand what each keyword does; the general structure of a query contains the table name, selected columns to retrieve and the condition to retrieve by.

SELECT column_name(s)
FROM table_name
WHERE condition;

In our example we did not have a WHERE condition but we will dig deeper on when to use it later on.

• SELECT: used to retrieve data.
• *: a wildcard expression used to indicate that we want to retrieve all the columns in our query.
• From table_name: the name of the table from which we want to retrieve our data.

Phew! that was a lot of information if you are still new to the world of databases & SQL, that's it for this blog and In the upcoming blog posts, we will start going deeper into SQL and provide an introduction to programming with python as part of our coding & data analysis journey!

Hope you enjoyed reading this blog as much as I enjoyed writing it. If you have any comments or suggestions please feel free to reach out ❤

This is the continuation in which we start by talking about Data Analysis. Don't worry we are still keeping things simple and will not introduce coding examples just yet.

What is Data Analysis?

As we have discussed in our previous blog, Data analysis is the process of cleaning, transforming, and modeling data to discover useful information that will help us make informed business decisions.

Why is Data Analysis gaining traction?

Data Analysis is not a new concept it has been around since the 1940's. One of the most famous examples is Henry Ford started measuring the speed of assembly lines, Analytics started becoming more mainstream when computers became a crucial part of the decision making process and providing more insights, with the development of Big data & the cloud; Data Analysis has started evolving dramatically. It involved gathering data from various sources, searching for patterns & creating data insights that help making much more informed decisions.

Data Analysis & Statistics

Data Analaysis is deeply intertwined with a wide variety of statistical methods; you don't have to worry if you have no background in statistics, but going forward if you want to pursue a further career; you will need to familiarize yourself with it to better know how to handle various types of data. Almost all libraries and tools already support built-in methods to handle different types of data.

Statistics deals with:

• Data Acquisition
• Data Interpretation
• Data Validation

An example for how statistics helps is if we have a supermarket. Let's assume that we want to identify what are our clients most frequently purchase, we can analyze the invoices and group the items by their frequency and Voila! we know now the best and worst performing items.

Role Of Data Analytics

Let's apply the supermarket example to what we are trying to explain in this section.

1. Gathering Hidden Insights: Analyzing customer's putchasing patterns.

2. Generating Reports: Getting the reports for how various items are performing to make better decisions by the management/decision makers for restocking or changing brands.

3. Market Analysis: Understanding the points of strength and weaknesses against our competitors and what is our store's edge.

Data Analysis Process

The Data Analysis process consists of Five Iterational steps:

• Identify What is the business requirement that we are trying to solve or what is the obstacle currently facing us?

• Collect the raw data that we will need to help answer the identified question. It can come from various sources as mentioned before and It can be like the sales invoices to our customers in the supermarket example.

• Clean the data. This often involves removing duplicates, inconsistencies & standardizing the format, and dealing with white spaces and other syntax errors.

• Analyze the data. By manipulating the data using various data analysis techniques and tools - That we will discuss in details later on -, you can begin to find trends & correlations that begin to tell a story about how our business is doing or how are our customers reacting to various changes.

• Interpret the results of your analysis to see how well you answered your original question and what recommendations can you make based on the data.

Tools used in Data Analysis

• R Programming: The leading tool for analytics, statistics & data modelling; It pre-installs all packages and runs on Unix, Windows & Mac OS.

• Python: Python is an open-source language that is easy to learn. We will discuss it in further blogs so if you have no background don't worry. Python has numerous packages that can aid you in your Data Analytics journey like: Sci-Kit Learn, Numpy & Pandas.

• Excel: One of the most famous Microsoft applications, It is mostly used for Internal sheets data & Comma separated files (CSV).

There are a plethora of tools when it comes to data analysis, some more advanced like:

• Apache Spark
• Tableau
• SAS

But for someone still getting to know the world of data analysis, you have come a long way and learned some of the basics that pave your way to start your own journey.

In the upcoming blog posts, we will dig deeper into SQL, Python & R and Implementing a small project to get our coding & data analysis journey started!

Hope you enjoyed reading this blog as much as I enjoyed writing it. If you have any comments or suggestions please feel free to reach out ❤