In this post, I'm going to create a simple, really simple, app that queries twitter, and then access the Sentiment140 REST API to analyze the tweet's content sentiment. After this, I'll count the results in a chart and also list the text content in a table.
Using the Sentiment140 REST API
The API is really simple to use. Since I'm lazy, I love simplicity. To use it, submit a text and it will return a simple JSON with a number value called polarity, which possible values are:- 0: negative
- 2: neutral
- 4: positive
$ curl -d "{'data': [{'text': 'I Love REST APIs'}]}" http://www.sentiment140.com/api/bulkClassifyJson
And it will return:
{"data":[{"text":"I Love REST APIs","polarity":4,"meta":{"language":"en"}}]}
See, it's not hard!
Querying Twitter
I wanted to query twitter, it was my plan... However, it's "new" API requires me to use OAuth to query. Twitter4j API, however, is great to handle Twitter, so I decided to keep twitter... See the code I use to query twitter:private ListsearchTwitter(String q){ Twitter twitter = new TwitterFactory().getInstance(); twitter.setOAuthConsumer(TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_KEY_SECRET); Query twitterQuery = new Query("lang:en " + q); try{ return twitter.search(twitterQuery) .getTweets().stream() // map to text and replace all special chars .map(s -> s.getText().replaceAll("[^a-zA-Z0-9\\s\\,\\.\\']+","")) .peek(System.out::println) .collect(Collectors.toList()); }catch(Exception e){ throw new RuntimeException(e); } }
Accessing the APIs with Java
The first thing to do is to use model what we want in a class and what is want is basically what the Sentiment140 API returns. So we created a Java bean named TextSentiment which code is below:@JsonIgnoreProperties(ignoreUnknown = true)
public class TextSentiment{
private String text;
private int polarity;
// getters setters and constructors
}
@XmlRootElement
public class TextSentimentList{
@XmlElement(name="data")
private List textSentiments;
// getters setters
}
Now, we have need to find a good way to send HTTP requests to this API and return instances of TextSentiment. To do this We created a class called TextSentimentService, and in this class we make use of JAX-RS 2 Client API to query Sentiment140, so we don't waste time parsing data... The method getSentiment140 was used to get the sentiment of all the Strings we passed
import org.jugvale.sentiments.model.TextSentimentList;
import org.jugvale.sentiments.model.TextSentiment;
import java.util.List;
import java.util.stream.Collectors;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
public class TextSentimentService{
private final String SENTIMENTS_BASE_URL = "http://www.sentiment140.com/api/bulkClassifyJson";
private final String MEDIA_TYPE = "application/json";
public TextSentimentList getSentiments(List texts){
TextSentimentList requestEntity = new TextSentimentList();
requestEntity.setTextSentiments(
texts.stream().map(TextSentiment::new).collect(Collectors.toList())
);
return ClientBuilder.newClient()
.target(SENTIMENTS_BASE_URL)
.request(MEDIA_TYPE)
.buildPost(Entity.entity(requestEntity, MEDIA_TYPE))
.invoke(TextSentimentList.class);
}
}
To access Twitter, we created a class SearchService, which has a method called search. In this method, we specify the provider we are wanting to retrieve texts to analyze, in our case, we have twitter:
import java.util.List;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import twitter4j.*;
import java.util.stream.Collectors;
public class SearchService{
/*
*It's mine!!! Please, use yours
* */
private final static String TWITTER_CONSUMER_KEY = "{KEY}";
private final static String TWITTER_CONSUMER_KEY_SECRET = "{SECRET}";
public static enum Provider{
TWITTER("Twitter");
private String name;
Provider(String name){
this.name = name;
}
public String toString(){
return this.name;
}
}
public List search(String q, Provider p){
switch (p){
case TWITTER:
return searchTwitter(q);
default:
throw new IllegalArgumentException("Provider \"" + p + "\" not implemented");
}
}
// searchTwitter method...
}
Notice that both Service classes can be extended to make use of another sentiment API or another social media API.
Testing our service classes
Before continuing with our application, let's make a simple test of the Sentiment Service API, see the code below:import org.junit.Test;
import org.jugvale.sentiments.service.*;
import org.jugvale.sentiments.model.*;
import java.util.Arrays;
public class TestService{
@Test
public void testServices(){
String TXT_1 = "obama is awesome";
String TXT_2 = "obama sucks";
String TXT_3 = "obama is eating a potato";
TextSentimentService s = new TextSentimentService();
TextSentimentList list = s.getSentiments(Arrays.asList(TXT_1, TXT_2, TXT_3));
list.getTextSentiments().stream().forEach(System.out::println);
//TODO: Create a Map with key equal the text and polarity is the value
}
}
With this small and simple test, we make sure we are accessing the API and returning something, so we also confirm the parsing of the response if correct.
The JavaFX APP
Our small APP contains one text field, one chart and one table! The user enters the query on the TextField, press enter, then our app takes that query to search the social media, get the results and then it sends the texts to the Sentiment140 API analyze. With the results, we update the user interface. See the code:import javafx.application.Application;
import javafx.scene.Scene;
import javafx.stage.Stage;
import javafx.collections.FXCollections;
import javafx.event.ActionEvent;
import javafx.scene.control.*;
import javafx.scene.control.cell.*;
import javafx.scene.layout.VBox;
import javafx.scene.chart.*;
import org.jugvale.sentiments.service.*;
import org.jugvale.sentiments.model.*;
import java.util.*;
public class App extends Application{
final TextSentimentService textSentimentService = new TextSentimentService();
final SearchService searchService = new SearchService();
final PieChart chart = new PieChart();
final TableView table = new TableView<>();
final TextField txtQuery = new TextField();
public static void main(String args[]){
launch(args);
}
@Override
public void start(Stage stage){
VBox root = new VBox(10);
root.getChildren().addAll(txtQuery, chart, table);
stage.setScene(new Scene(root));
stage.setWidth(400);
stage.setHeight(550);
stage.setTitle("Sentiments");
stage.show();
txtQuery.setOnAction(e -> {
List textSentiments = getTextSentiments(txtQuery.getText());
fillTable(textSentiments);
fillChart(textSentiments);
chart.setTitle("Sentiments for \"" + txtQuery.getText() +"\"");
});
txtQuery.setPromptText("Enter your query for text and press enter");
initializeApp();
}
private void initializeApp(){
txtQuery.setText(null);
txtQuery.getOnAction().handle(null);
addTableColumns();
}
private void addTableColumns(){
TableColumn textCol = new TableColumn<>("Text");
textCol.setCellValueFactory(new PropertyValueFactory("text"));
textCol.setMinWidth(200);
TableColumn polarityCol = new TableColumn<>("Polarity");
polarityCol.setCellValueFactory(new PropertyValueFactory("polarity"));
polarityCol.setMinWidth(80);
polarityCol.setResizable(false);
table.getColumns().setAll(textCol, polarityCol);
}
public void fillTable(List textSentiments){
table.setItems(FXCollections.observableArrayList(textSentiments));
}
public void fillChart(List textSentiments){
long polarity0Count = textSentiments.stream().filter(s -> s.getPolarity() == 0).count();
long polarity2Count = textSentiments.stream().filter(s -> s.getPolarity() == 2).count();
long polarity4Count = textSentiments.stream().filter(s -> s.getPolarity() == 4).count();
chart.setData(
FXCollections.observableArrayList(
new PieChart.Data(":(", polarity0Count),
new PieChart.Data(":|", polarity2Count),
new PieChart.Data(":)", polarity4Count)
));
}
public List getTextSentiments(String q){
return textSentimentService.getSentiments(query(q)).getTextSentiments();
}
public List query(String q){
if(q == null){
return Arrays.asList("obama is awesome", "obama sucks", "obama eats potato");
}else{
// TODO: add new providers and adapt the app to support it
return searchService.search(q, SearchService.Provider.TWITTER);
}
}
}
Notice we use Java 8 features to deal with the List of TextSentiment objects! Also, we used Lambda in some places of the code, can you spot it?
Do you wanna play with it?
The app code is on github and it uses Maven. if you want to play with it, use git app to clone the repository locally and build it. Remember to use Java 8 and Maven 3.x! Steps:
Clone the APP
$ git clone https://github.com/jesuino/sentiments-app.git
Go to the app root directory and use the following command to build:
$ cd sentiments-app
$ mvn clean package
You can also run a test:
$ mvn test
Finally, you can run the App using the run.sh script. Remember to set JAVA_HOME correctly! run.sh content:
export JAVA_HOME=/opt/java/jdk1.8.0/
mvn package exec:java -Dexec.mainClass="org.jugvale.sentiments.view.App" -DskipTests
App screenshots
Searching for happy words:
Searching for sad words:
Conclusion
That's all, folks. Just wanted to share this small fun app with you. I made the code really simple- Extending by adding other text sentiment analyzer. Lots of text sentiments APIs here.
- Extending by adding other source then twitter, for example: news sites, google plus, facebook etc.
- Improve the view by adding more controls, a "waiting screen", etc
- Add a CSS to improve the look of the app
Nenhum comentário:
Postar um comentário