Author Archives: Mohamed Sanaulla

Using Gmail as SMTP server from Java, Spring Boot apps

Gmail users can use Gmail’s SMTP server smtp.gmail.com to send emails from their Spring Boot apps. For this let us do some setup in the app:

  1. Provide SMTP connection properties in the application.properties file:
    spring.mail.host=smtp.gmail.com
    spring.mail.username=<your gmail/google app email>
    spring.mail.password=*****
    spring.mail.port=587
    spring.mail.properties.mail.smtp.starttls.enable=true
    spring.mail.properties.mail.smtp.auth=true
    spring.mail.properties.mail.smtp.starttls.required=true
  2. Use Spring Boot Email tools library – which is a wrapper over Spring Boot Email starter library.  Add the following in your pom.xml:
    <dependency>
        <groupId>it.ozimov</groupId>
        <artifactId>spring-boot-email-core</artifactId>
        <version>0.6.3</version>
    </dependency>
  3. Annotation your application’s main class (i.e class annotated with @SpringBootApplication) with @EnableEmailTools:
    @SpringBootApplication
    @EnableEmailTools
    public class EmailApplication {
        public static void main(String[] args){
            SpringApplication.run(EmailApplication.class, args);
        }
    }
  4. Let’s write a test which uses it.ozimov.springboot.mail.service.EmailService bean to send an email:
    @RunWith(SpringRunner.class)
    @SpringBootTest
    public class EmailServiceTest {
        @Autowired 
        it.ozimov.springboot.mail.service.EmailService emailService; 
    
        @Value("${spring.mail.username}") String fromEmail; 
        @Test 
        public void testSendEmail() throws UnsupportedEncodingException { 
            User user = new User(); 
            user.setEmail("sanaulla123@gmail.com"); 
            user.setDisplayName("Mohamed Sanaulla"); 
            final Email email = DefaultEmail.builder() 
                .from(new InternetAddress(fromEmail, "From Name"))
                .to(Lists.newArrayList(new InternetAddress(
                    user.getEmail(), user.getDisplayName()))) 
                .subject("Testing email")
                .body("Testing body ...")
                .encoding("UTF-8").build();
            emailService.send(email); 
        }
    }

Continue reading

Advertisements

Getting to know about java.nio.file.Path – 2

In Part 1 of this, we looked at most of the APIs in the java.nio.file.Path class. In this article, we will look at the remaining APIs.

Using register()

This API allows us to register an implementation of java.nio.file.WatchService interface which will listen for events like directory creation, modification, and deletion. And it intimates the listeners by means of a java.nio.file.WatchKey. I would like to dedicate a different article for this API because it involves another new feature which was introduced in Java 7.

Using resolve()

This method deals with two Path instances. One instance on this the resolve() method is called and the other instance which is passed as an argument. The argument can be either a Path instance or a String representing the path.

This method resolves the other path against this path. The resolution is done as follows:

  1. If the other path is an absolute path, then it returns the other path. Because the other path can be reached by using the absolute path.
  2. If the other path is a relative path then the other path is appended to this path. For example:
    Path path = Paths.get("src", "main", "resources");
    Path other = Paths.get("blogsamples");
    
    assertThat(path.resolve(other)).isEqualTo(
            Paths.get("src", "main", "resources", "blogsamples"));

The different scenarios in which this method can be invoked is given in the test below:

@Test
public void testResolved() throws IOException {
    Path path = Paths.get("src", "main", "resources");
    Path other = Paths.get("blogsamples");

    assertThat(path.resolve(other)).isEqualTo(
            Paths.get("src", "main", "resources", "blogsamples"));

    other = Paths.get("/Users");
    assertThat(path.resolve(other)).isEqualTo(Paths.get("/Users"));

    path = Paths.get("/src", "main", "resource");
    assertThat(path.resolve("/Users")).isEqualTo(Paths.get("/Users"));
}

Using resolveSibling()

This method is similar to resolve() except that it considers this path’s parent to resolve the other path. Again there are different possibilities which I have captured in the test below:

@Test
public void testResolveSibling(){
    Path path = Paths.get("src", "main", "resources", "test1");
    Path other = Paths.get("test2");

    //both paths are not absolute
    assertThat(path.resolveSibling(other)).isEqualTo(
            Paths.get("src", "main", "resources", "test2"));

    //other path is absolute
    assertThat(path.resolveSibling("/test2")).isEqualTo(
        Paths.get("/test2"));

    //this path has no parent
    path = Paths.get("/");
    assertThat(path.resolveSibling("/test2")).isEqualTo(
        Paths.get("/test2"));

    //the other path is empty and this path has no parent
    assertThat(path.resolveSibling("")).isEqualTo(Paths.get(""));

    //the other path is empty and this path has parent
    path = Paths.get("src", "main", "resources", "test1");
    assertThat(path.resolveSibling("")).isEqualTo(
            Paths.get("src", "main", "resources"));
}

Using relativize()

This method returns a relative path that when resolved against this path returns the other path (i.e the path which is passed as a parameter).

I have tried to illustrate in the tests below the different possibilities while trying to create a relative path between two paths.

Path path = Paths.get("src", "main", "resources", "test1");
Path other = Paths.get("test2");

assertThat(path.relativize(other).toString())
    .isEqualTo("..\\..\\..\\..\\test2");

In the above case, both the paths are relative. It requires 4 hops backward from the src/main/resources/test1 to reach /test2. The same is obtained by applying the relativize method.

If one of the paths is absolute and the other is relative, then invoking relativize results in an IllegalArgumentException as shown below:

@Test(expected = IllegalArgumentException.class)
public void testRelativize_WithRelativeAndAbsolutePath(){
    Path path = Paths.get("/src", "main", "resources", "test1");
    Path other = Paths.get("src", "main", "resources");
    path.relativize(other);
}

If both the paths are absolute, then the output of relativize() is implementation dependent. The below test is written against JDK 8 on Windows platform:

@Test
public void testRelativize_WithAbsolutePaths(){
    Path path = Paths.get("/src", "main", "resources", "test1");
    Path other = Paths.get("/src", "main", "resources", "test1", "test2");
    assertThat(path.relativize(other).toString())
        .isEqualTo("test2");
}

Using startsWith()

This method checks if the path on which the startsWith() method has the same name elements in the beginning as that of the path passed as the argument. And the path passed as the argument has no extra name elements that are not present in this path.

For example: /a/b/c starts with /a/b , a/b/c/d starts with a/b/c

Let us look at the different possible cases while invoking the method:

@Test
public void testStartsWith(){
    //both paths are absolute
    Path path = Paths.get("/src", "main", "resources", "test1");
    Path other = Paths.get("/src", "main", "resources");
    assertThat(path.startsWith(other)).isTrue();

    /*
    both paths are absolute, where as the other 
    path has more name elements 
    */
    path = Paths.get("/src", "main", "resources", "test1");
    other = Paths.get("/src", "main", "resources", 
        "test1", "test2");
    assertThat(path.startsWith(other)).isFalse();

    //both paths are same
    path = Paths.get("/src", "main", "resources", "test1");
    other = Paths.get("/src", "main", "resources", "test1");
    assertThat(path.startsWith(other)).isTrue();

    //either of them is relative
    path = Paths.get("src", "main", "resources", "test1");
    other = Paths.get("/src", "main", "resources", "test1");
    assertThat(path.startsWith(other)).isFalse();

    //both of them are relative
    path = Paths.get("src", "main", "resources", "test1");
    other = Paths.get("src", "main", "resources");
    assertThat(path.startsWith(other)).isTrue();

}

Installing Git and EPEL Packages on RHEL 5.*

I was looking to install Git on a Linux server running RHEL 5.*. And using the EPEL repos helped me to locate the required Git rpm along with its dependencies.

I followed the following steps to install Git:

wget http://archives.fedoraproject.org/pub/archive/epel/epel-release-latest-5.noarch.rpm

rpm -ivh epel-release-latest-5.noarch.rpm

yum install git

Getting to know about java.nio.file.Path – 1

Introduction

The last few released of Java namely Java 7, Java 8 and the upcoming Java 9 have quite a lot of features which makes the life of Java developers easier. (I know Java 9 will make it tougher, but only while you adopt the new paradigm. After that it’s going to be much better).

One of the features or API was the enhancements of the File API introduced in Java 7. One of the new classes of that feature set is java.nio.file.Path and its factory java.nio.file.Paths.

Maven Dependencies

We will be using JUnit and AssertJ to write our tests to demonstrate the API.

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
</dependency>
<dependency>
    <groupId>org.assertj</groupId>
    <artifactId>assertj-core</artifactId>
    <!-- use 2.8.0 for Java 7 projects -->
    <version>3.8.0</version>
    <scope>test</scope>
</dependency>

Creating an instance of java.nio.file.Path

As I mentioned before java.nio.file.Paths is the creator for java.nio.file.Path that is it provides two factory methods:

  • static Path get(String first, String … more)
  • static Path get(URI uri)

that can be used to get an instance of java.nio.file.Path. Let us look at the two ways to obtain the instance:

@Test
public void testPathCreation(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.isAbsolute()).isFalse();
    assertThat(path.toString()).isEqualTo("src\\main\\resources");

    Path uriPath = Paths.get(URI.create("file:///Users/Mohamed/git"));
    assertThat(uriPath.isAbsolute()).isFalse();
    assertThat(uriPath.toAbsolutePath().toString())
            .isEqualTo("C:\\Users\\Mohamed\\git");
}

Exploring the APIs

Using endsWith()

This method is used to check if a given Path object ends with another Path object or a path represented as a String object.

@Test
public void testEndsWith(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.endsWith(Paths.get("main","resources"))).isTrue();
    assertThat(path.endsWith("resources")).isTrue();
}

Using getFileName()

This method returns the name of the directory or the file present at the terminal or end of the path.

@Test
public void testGetFileName(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.getFileName().toString()).isEqualTo("resources");
    path = Paths.get("src", "test", "java", "info", 
        "sanaulla","PathDemoTest.java");
    assertThat(path.getFileName().toString())
        .isEqualTo("PathDemoTest.java");
}

Using getFileSystem()

This method returns an instance of java.nio.file.FileSystem representing the underlying file system. We will look at this in detail in a future post.

@Test
public void testGetFileSystem(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.getFileSystem()).isNotNull();
    assertThat(path.getFileSystem().getSeparator()).isEqualTo("\\");
    path.getFileSystem().getRootDirectories().forEach(System.out::println);
}

Using getName() and getNameCount()

The getNameCount() returns number of name components present in the path where each name component is seperated by a file seperator. And the method getName() takes an index and returns the name component at the index.

For example  a given path: /var/log/myapp/spring.log has 4 name components and the component position is 0 based. So the name component at index 1 is log.

@Test
public void testGetName(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.getName(0)).isEqualTo(Paths.get("src"));

    assertThat(path.getName(path.getNameCount() - 1))
            .isEqualTo(Paths.get("resources"));
}

Using getParent()

This API returns the path from the root of a path until the terminal directory or file (i.e excluding it). For example: invoking getParent() on a Path instance representing /var/log/myapp/spring.log returns a Path instance representing /var/log/myapp

It returns null if the given path has no parent or if it is the root directory.

@Test
public void testGetParent(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.getParent()).isEqualTo(Paths.get("src", "main"));
    assertThat(Paths.get("/").getParent()).isNull();
}

Using getRoot()

This API returns a Path instance of the root if it exists or null for a given instance of Path.

@Test
public void testGetRoot(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.getRoot()).isNull();

    path = Paths.get("/users", "Mohamed", "git", "blogsamples");
    assertThat(path.getRoot()).isEqualTo(Paths.get("/"));
}

Using normalize()

This API is a bit tricky. It removes redundant elements in your path. Redundant elements are those whose removal will eventually result in a similar Path. For example: if we have a path src\..\src\main\java is equivalent to src\main\java. The normalize() API helps in achieving the latter from the former.

@Test
public void testNormalize(){
    Path path = Paths.get("src","..", "src", "main", "resources", ".");
    assertThat(path.toString())
            .isEqualTo("src\\..\\src\\main\\resources\\.");
    assertThat(path.normalize().toString())
            .isEqualTo("src\\main\\resources");
}

Using subpath()

This method returns a sub path identified by the lower bound and upper bound which are passed as parameters to the method. The upper bound is excluded while computing the sub path.

@Test
public void testSubpath(){
    Path path = Paths.get("Mohamed", "git", 
            "blogsamples", "src", "main", "resources");
    assertThat(path.subpath(2, 3).toString()).isEqualTo("blogsamples");
    assertThat(path.subpath(0, path.getNameCount()).toString())
            .isEqualTo("Mohamed\\git\\blogsamples\\src\\main\\resources");
}

Using toAbsolutePath()

This method returns the absolute path for the given path. An absolute path originates from the root of the file system.

@Test
public void testToAbsolutePath(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.toAbsolutePath().toString())
       .isEqualTo("C:\\Users\\Mohamed\\git\\blogsamples\\src\\main\\resources");
}

Using toFile()

This is a very handy way to create an instance of java.io.File. We can leverage the use of creating a Path object with multiple folder levels and then use toFile() to get an instance of File.

@Test
public void testToFile(){
    Path path = Paths.get("src", "main", "resources");
    File file = path.toFile();
    assertThat(file).isNotNull();
    assertThat(file.isDirectory()).isTrue();
    assertThat(file.exists()).isTrue();
}

Using toRealPath()

This method can be used to resolve a symbolic link to its real location. To test this API we create a symbolic link:

On windows you would use:

mklink /D "C:\blogsample" "C:\Users\Mohamed\git\blogsamples"

On Linux, you would use

ln -s /var/log/sample.log sample

The method takes an option of type LinkOption. As of now, this enum has one element i.e NOFOLLOW_LINKS. If this option is passed then the symbolic link is not resolved to its real path.

@Test
public void testToRealPath() throws IOException {
    Path path = Paths.get( "/blogsample");
    assertThat(path.toRealPath().toString())
            .isEqualTo("C:\\Users\\Mohamed\\git\\blogsamples");
    assertThat(path.toRealPath(LinkOption.NOFOLLOW_LINKS).toString())
            .isEqualTo("C:\\blogsample");
}

Using toUri()

This method returns a URI representation of a given path. Generally, on Windows, you would see something of the form: file:///C:/. But this is system dependent

@Test
public void testToUri(){
    Path path = Paths.get("src", "main", "resources");
    assertThat(path.toUri()).isEqualTo(
      URI.create("file:///C:/Users/Mohamed/git/blogsamples/src/main/resources/"));
}

Note: It’s important to note that the return type of most of the APIs is an instance of java.nio.file.Path. This helps us in chaining multiple methods and invoke them on a single java.nio.file.Path instance.

In the next article, we will look at the remaining few APIs in java.nio.file.Path.

Index Data from a Relational Database into Elasticsearch – 2

In my previous article, we showed a straight forward way to index data from the relational database into Elasticsearch. In this article we will show you:

  1. Create mappings for the index
  2. Understand the SQL syntax for creating nested objects and nested object arrays in the elasticsearch

Continue reading

Index Data from a Relational Database into Elasticsearch – 1

Elasticsearch provides powerful search capabilities with support for sharding and replication of the data. So we would want to index data available in our DB into Elasticsearch.

There are multiple ways to index data into Elasticsearch:

  1. Use Logstash to setup source as DB and sink as Elasticsearch and use a filter if required to build JSON object.
  2. Use an external library elasticsearch-jdbc which runs, in its own process, external to Elasticsearch instance. It makes use of the transport client and its bulk APIs to index data into Elasticsearch.

In this article, we will look at the approach 2 i.e using an external library running as a separate process.

Continue reading

Shell Script to Detect If the IP Address Is Googlebot

1. Introduction

Google has explained here on how to verify if the given IP belongs to Googlebot. So instead of hard-coding all the IPs which might change eventually, Google has suggested doing a DNS lookup using host command on Linux. The steps suggested in the article are:

  1. reverse DNS lookup using the IP to get domain name
  2. check if the domain name contains googlebot.com or google.com. The types of bots and their names can be found here.
  3. forward DNS lookup using the domain name obtained in step 1 to get the IP and verify that this IP is same as the IP you initially started with.

2. Implementation Approach

And I wanted to check if the IPs ( I had around 45) belonged to google bot. One option was to run host for each of the IPs, as suggested in the above steps, but this was practically not possible and if possible it would be time-consuming. So I came up with a simple shell script to do this job.

  1. reverse DNS lookup to get the domain name
    #hostName is the complete domain name
    hostName = host $line | cut -d" " -f 5

    So a sample response of host command would be 246.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-246.googlebot.com. And we can extract the domain name using the cut command as shown above. This gives us hostName = crawl-66-249-66-246.googlebot.com.

  2. forward DNS lookup to get the IP
    hostIp=host $hostName | cut -d" " -f 4

    A sample response to the host command, in this case, would be: crawl-66-249-66-246.googlebot.com has address 66.249.66.246. And we extract the IP using the cut command shown above which gives us the hostIp = 66.249.66.246.

  3. verify the domain name to contain googlebot and the IP obtained in step 2 is same as the IP we started with in step 1
    if [ $line == $hostIp ] && [ $domainName == "googlebot.com" ]
    then
        echo "Googlebot: $hostIp -> $hostName"
    fi

3. Complete Shell Script

Let the IPs be in the file googlebots, for example, let us add these IPs:

66.249.66.246
66.249.66.97
66.249.64.12

The shell script is given below:

#!/bin/sh
file="googlebots"
while read -r line
do
    hostName=`host $line | cut -d" " -f 5`
    domainName=`echo $hostName | cut -d"." -f2,3`
    #echo $domainName
    #echo "$hostName"
    hostIp=`host $hostName | cut -d" " -f 4`
    #echo "$hostIp"
    if [ $line == $hostIp ] && [ $domainName == "googlebot.com" ]
    then
        echo "Googlebot: $hostIp -> $hostName"
    fi
done < "$file"