Accumulo - Examples/Java API
1.
Use the following command to prompt
the client shell:
bin/accumulo shell -u root
Then enter the password for your Accumulo instance in our example
we set the instance name and password to accum/accum.
2.
You may type help for the list of
available commands.
3.
You can create a new table using:
createtable command:
4.
E.g: create a new table called
usertable: "createtable
usertable"
·
The following running
examples are from apache accumulo
5.
Creating
a new user:
A user
does not by default have permission to create a table.
root@instance> createuser username
Enter new password for 'username': ********
Please confirm new password for 'username':
********
root@instance> user username
Enter password for user username: ********
username@instance> createtable usertable
06 10:48:47,931
[shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException:
Error PERMISSION_DENIED - User does not have permission to perform this action
username@instance> userpermissions
System permissions:
Table permissions (!METADATA): Table.READ
username@instance>
6.
Granting
permissions to a user:
username@instance> user root
Enter password for user root: ********
root@instance> grant -s System.CREATE_TABLE -u
username
root@instance> user username
Enter password for user username: ********
username@instance> createtable usertable
username@instance> userpermissions
System permissions: System.CREATE_TABLE
Table permissions (!METADATA): Table.READ
Table permissions
(usertable): Table.READ, Table.WRITE, Table.BULK_IMPORT, Table.ALTER_TABLE,
Table.GRANT, Table.DROP_TABLE
username@instance usertable>
7.
Inserting
data with visibilities:
usage: insert <row> <colfamily> <colqualifier> <value> [-?] [-l <expression>] [-t <timestamp>]
description: inserts a record
-?,-help display this help
-l,-authorization-label <expression> formatted authorization label expression
-t,-timestamp <timestamp> timestamp to use for insert.
Visibilities are
boolean AND (&) and OR (|) combinations of authorization tokens.
Authorization tokens are arbitrary strings taken from a restricted ASCII
character set. Parentheses are required to
specify order of operations in visibilities.
username@instance usertable> insert row f1 q1
v1 -l A
username@instance usertable> insert row f2 q2
v2 -l A&B
username@instance usertable> insert row f3 q3
v3 -l apple&carrot|broccoli|spinach
06 11:19:01,432
[shell.Shell] ERROR: org.apache.accumulo.core.util.BadArgumentException: cannot
mix | and & near index 12 apple&carrot|broccoli|spinach ^
username@instance
usertable> insert row f3 q3 v3 -l (apple&carrot)|broccoli|spinach
username@instance usertable>
8.
Scanning
with authorizations:
usage: scan [-?] [-b <start-row>] [-c <<columnfamily>[:<columnqualifier>]>] [-e
<end-row>] [-np] [-s <comma-separated-authorizations>] [-st]
description: scans the table, and displays the resulting records
-?,-help display this help
-b,-begin-row <start-row> begin row (inclusive)
-c,-columns <<columnfamily>[:<columnqualifier>]> comma-separated columns
-e,-end-row <end-row> end row (inclusive)
-np,-no-pagination disables pagination of output
-s,-scan-authorizations <comma-separated-authorizations> scan authorizations
-st,-show-timestamps enables displaying timestamps
Authorizations are sets of authorization tokens.
Each Accumulo user has authorizations and each Accumulo scan has
authorizations. Scan authorizations are only allowed to be a subset of the
user's authorizations. By default, a user's authorizations set is empty.
username@instance usertable> scan
username@instance usertable> scan -s A
06 11:43:14,951
[shell.Shell] ERROR: java.lang.RuntimeException:
org.apache.accumulo.core.client.AccumuloSecurityException: Error
BAD_AUTHORIZATIONS - The user does not have the specified authorizations
assigned
username@instance usertable>
9.
Setting
authorizations for a user:
usage: setauths [-?] -c | -s <comma-separated-authorizations> [-u <user>]
description: sets the maximum scan authorizations for a user
-?,-help display this help
-c,-clear-authorizations clears the scan authorizations
-s,-scan-authorizations <comma-separated-authorizations> set the scan
authorizations
-u,-user <user> user to operate on
username@instance usertable> setauths -s A
06 11:53:42,056
[shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException:
Error PERMISSION_DENIED - User does not have permission to perform this action
username@instance usertable>
A user cannot set
authorizations unless the user has the System.ALTER_USER permission. The root
user has this permission.
username@instance usertable> user root
Enter password for user root: ********
root@instance usertable> setauths -s A -u
username
root@instance usertable> user username
Enter password for user username: ********
username@instance usertable> scan -s A
row f1:q1 [A]
v1
username@instance usertable> scan
row f1:q1 [A]
v1
username@instance usertable>
The default authorizations for a scan are the
user's entire set of authorizations.
username@instance usertable> user root
Enter password for user root: ********
root@instance usertable> setauths -s
A,B,broccoli -u username
root@instance usertable> user username
Enter password for user username: ********
username@instance usertable> scan
row f1:q1 [A]
v1
row f2:q2 [A&B] v2
row f3:q3
[(apple&carrot)|broccoli|spinach]
v3
username@instance usertable> scan -s B
username@instance usertable>
If you
want, you can limit a user to only be able to insert data which they can read
themselves. It can be set with the following constraint.
username@instance usertable> user root
Enter password for user root: ******
root@instance
usertable> config -t usertable -s
table.constraint.1=org.apache.accumulo.core.security.VisibilityConstraint
root@instance usertable> user username
Enter password for user username: ********
username@instance usertable> insert row f4 q4
v4 -l spinach
Constraint
Failures:
ConstraintViolationSummary(constrainClass:org.apache.accumulo.core.security.VisibilityConstraint,
violationCode:2, violationDescription:User does not have authorization on
column visibility, numberOfViolatingMutations:1)
username@instance usertable> insert row f4 q4
v4 -l spinach|broccoli
username@instance usertable> scan
row f1:q1 [A]
v1
row f2:q2 [A&B] v2
row f3:q3
[(apple&carrot)|broccoli|spinach]
v3
row f4:q4 [spinach|broccoli] v4
username@instance usertable>
Java API
1.
Row operations – All basic row operations createtable/insert/delete etc
/*
* Author
: Amal Babu
*
Description : Row operations include create,insert,delete in JAVA API
* */
package com.amal.accumulo;
import java.io.IOException;
import java.util.Map.Entry;
import org.apache.accumulo.core.Constants;
import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.Connector;
import org.apache.accumulo.core.client.MutationsRejectedException;
import org.apache.accumulo.core.client.Scanner;
import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.client.ZooKeeperInstance;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
/**
* A
demonstration of reading entire rows and deleting entire rows.
*/
public class RowOperations {
private static final Logger log = Logger.getLogger(RowOperations.class);
private static Connector connector;
private static String table = "accumulotable";
private static BatchWriter bw;
public static void main(String[] args) throws
AccumuloException, AccumuloSecurityException, TableExistsException,
TableNotFoundException,
MutationsRejectedException {
if (args.length != 4) {
log.error("Usage:
<instance name> <zoo keepers> <username>
<password>");
return;
}
// First the setup work
connector = new ZooKeeperInstance(args[0], args[1]).getConnector(args[2],
args[3].getBytes());
// lets create an example table
connector.tableOperations().create(table);
// lets create 3 rows of information
Text row1 = new Text("row1");
Text row2 = new Text("row2");
Text row3 = new Text("row3");
// Which means 3 different mutations
Mutation
mut1 = new Mutation(row1);
Mutation
mut2 = new Mutation(row2);
Mutation
mut3 = new Mutation(row3);
// And we'll put 4 columns in each row
Text col1 = new Text("1");
Text col2 = new Text("2");
Text col3 = new Text("3");
Text col4 = new Text("4");
// Now we'll add them to the mutations
mut1.put(new Text("column"), col1, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut1.put(new Text("column"), col2, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut1.put(new Text("column"), col3, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut1.put(new Text("column"), col4, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut2.put(new Text("column"), col1, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut2.put(new Text("column"), col2, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut2.put(new Text("column"), col3, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut2.put(new Text("column"), col4, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut3.put(new Text("column"), col1, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut3.put(new Text("column"), col2, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut3.put(new Text("column"), col3, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
mut3.put(new Text("column"), col4, System.currentTimeMillis(), new Value("This is
the value for this key".getBytes()));
// Now we'll make a Batch Writer
bw = connector.createBatchWriter(table, 100000l, 30l, 1);
// And add the mutations
bw.addMutation(mut1);
bw.addMutation(mut2);
bw.addMutation(mut3);
// Force a send
bw.flush();
// Now lets look at the rows
Scanner
rowThree = getRow(new Text("row3"));
Scanner
rowTwo = getRow(new Text("row2"));
Scanner
rowOne = getRow(new Text("row1"));
// And print them
log.info("This is
everything");
printRow(rowOne);
printRow(rowTwo);
printRow(rowThree);
System.out.flush();
// Now lets delete rowTwo with the iterator
rowTwo = getRow(new Text("row2"));
deleteRow(rowTwo);
// Now lets look at the rows again
rowThree = getRow(new Text("row3"));
rowTwo = getRow(new Text("row2"));
rowOne = getRow(new Text("row1"));
// And print them
log.info("This is
row1 and row3");
printRow(rowOne);
printRow(rowTwo);
printRow(rowThree);
System.out.flush();
// Should only see the two rows
// Now lets delete rowOne without passing in the
iterator
deleteRow(row1);
// Now lets look at the rows one last time
rowThree = getRow(new Text("row3"));
rowTwo = getRow(new Text("row2"));
rowOne = getRow(new Text("row1"));
// And print them
log.info("This is
just row3");
printRow(rowOne);
printRow(rowTwo);
printRow(rowThree);
System.out.flush();
// Should only see rowThree
// Always close your batchwriter
bw.close();
// and lets clean up our mess
connector.tableOperations().delete(table);
// fin~
}
/**
*
Deletes a row given a text object
*
* @param row
* @throws TableNotFoundException
* @throws AccumuloSecurityException
* @throws AccumuloException
*/
private static void deleteRow(Text row) throws
AccumuloException, AccumuloSecurityException, TableNotFoundException {
deleteRow(getRow(row));
}
/**
*
Deletes a row, given a Scanner of JUST that row
*
* @param scanner
*/
private static void deleteRow(Scanner scanner) throws MutationsRejectedException {
Mutation
deleter = null;
// iterate through the keys
for (Entry<Key,Value> entry : scanner) {
// create a mutation for the row
if (deleter == null)
deleter
= new Mutation(entry.getKey().getRow());
// the remove function adds the key with the
delete flag set to true
deleter.putDelete(entry.getKey().getColumnFamily(),
entry.getKey().getColumnQualifier());
}
bw.addMutation(deleter);
bw.flush();
}
/**
* Just
a generic print function given an iterator. Not necessarily just for printing a
single row
*
* @param scanner
*/
private static void printRow(Scanner scanner) {
// iterates through and prints
for (Entry<Key,Value> entry : scanner)
log.info("Key:
" + entry.getKey().toString() +
" Value: " + entry.getValue().toString());
}
/**
* Gets
a scanner over one row
*
* @param row
* @return
* @throws TableNotFoundException
* @throws AccumuloSecurityException
* @throws AccumuloException
* @throws IOException
*/
private static Scanner getRow(Text row) throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
// Create a scanner
Scanner
scanner = connector.createScanner(table, Constants.NO_AUTHS);
// Say start key is the one with key of row
// and end key is the one that immediately
follows the row
scanner.setRange(new Range(row));
return scanner;
}
}
How
to execute ?
·
Make it as runnable jar
and copy it to /accumulo/lib/ext folder.
·
/accumulo/bin/accumulo
com.amal.accumulo.RowOperations
2.
Adding cell level security
– JAVA API
/*
* Author
: Amal Babu
*
Description : Row operations include create,insert,delete in JAVA API
* */
package com.amal.accumulo;
import java.io.IOException;
import java.util.Map.Entry;
import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.Connector;
import org.apache.accumulo.core.client.MutationsRejectedException;
import org.apache.accumulo.core.client.Scanner;
import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.client.ZooKeeperInstance;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
import org.apache.accumulo.core.security.ColumnVisibility;
import org.apache.hadoop.io.Text;
import org.apache.log4j.Logger;
public class CellLevelSecurity {
private static String table;
private static final Logger log = Logger.getLogger(RowOperations.class);
private static Connector connector;
// private static String table = "accumulotable";
private static BatchWriter bw;
// default value
private static String securityLevelsWhichNeedsToAccess;
public static void main(String[] args) throws
AccumuloException,
AccumuloSecurityException,
TableExistsException,
TableNotFoundException,
MutationsRejectedException {
if (args.length != 6) {
log.error("Usage:
<instance name> <zoo keepers> <username> <password>
<accumulotablename> <securitylevel>");
return;
}
// First the setup work
connector = new ZooKeeperInstance(args[0], args[1]).getConnector(
args[2],
args[3].getBytes());
table = args[4];
securityLevelsWhichNeedsToAccess = args[5];
String[]
arrayAuthorizations = securityLevelsWhichNeedsToAccess
.split(",");
// listOfAccessQualifiers = new
//
ArrayList<String>(Arrays.asList(splitString));
// lets create an example table
connector.tableOperations().create(table);
System.out.println("Table
created successfully...!!");
// lets create 3 rows of information
Text
row1 = new Text("row1");
Text
row2 = new Text("row2");
Mutation
mut1 = new Mutation(row1);
Mutation
mut2 = new Mutation(row2);
Text
colFam_name = new Text("Name");
Text
colQual_name = new Text("Identifier");
ColumnVisibility
colVis_name = new ColumnVisibility("public");
// Now we'll add them to the mutations
mut1.put(colFam_name,
colQual_name, colVis_name,
new Value("Amal".getBytes()));
Text
colFam_pass = new Text("Password");
Text
colQual_pass = new Text("Secret");
ColumnVisibility
colVis_pass = new ColumnVisibility("private");
// Now we'll add them to the mutations
mut2.put(colFam_pass,
colQual_pass, colVis_pass,
new Value("amal@123".getBytes()));
// Now we'll make a Batch Writer
bw = connector.createBatchWriter(table, 100000l, 30l, 1);
// And add the mutations
bw.addMutation(mut1);
bw.addMutation(mut2);
// Force a send
bw.flush();
// Now lets look at the rows
Scanner
rowOne = getRow(new Text("row1"), arrayAuthorizations);
Scanner
rowTwo = getRow(new Text("row2"), arrayAuthorizations);
// And print them
log.info("This is
everything");
printRow(rowOne);
printRow(rowTwo);
System.out.flush();
bw.close();
// and lets clean up our mess
connector.tableOperations().delete(table);
}
/**
* Just a generic print function given an
iterator. Not necessarily just for
* printing a single row
*
* @param scanner
*/
private static void printRow(Scanner scanner) {
// iterates through and prints
for (Entry<Key, Value> entry : scanner)
log.info("Key:
" + entry.getKey().toString() +
" Value: "
+
entry.getValue().toString());
}
/**
* Gets a scanner over one row
*
* @param row
* @return
* @throws
TableNotFoundException
* @throws
AccumuloSecurityException
* @throws
AccumuloException
* @throws IOException
*/
private static Scanner getRow(Text row, String[] authorizations)
throws AccumuloException, AccumuloSecurityException,
TableNotFoundException
{
// Create a scanner
Authorizations
auth = new Authorizations(authorizations);
Scanner
scanner = connector.createScanner(table, auth);
// Say start key is the one with key of row
// and end key is the one that immediately
follows the row
scanner.setRange(new Range(row));
return scanner;
}
}
How
to execute ?
·
Make it as runnable jar
and copy it to /accumulo/lib/ext folder.
·
/accumulo/bin/accumulo
com.amal.accumulo.CellLevelSecurity <instance name> <zoo
keepers> <username> <password> <accumulotablename>
<securitylevel>
