Preface

Imagine that you have a set of tasks to accomplish today written down on a piece of paper, ordered by their importance. As the day goes on: you’ll mark some tasks as complete, maybe one task becomes extra important and you move it to the top of the list, or maybe you will discover a new task that needs done and add it to the bottom or middle of the list.

There is a hidden difficulty in keeping ordered data sets (like tasks ordered by importance or priority) in a traditional sql database. In this post I hope to explain well the decisions that I made while building a Flashcard app.

Available Strategies

Position as a Field

In this strategy, your table has a column that holds a position value. Any time you need to change the position of a single record, you will also need to update all records with higher/lower position.

• Pros:
  • Reading an ordered sequence of records from the database is trivial and performant.
• Cons:
  • Updating records in a way that keeps order is not trivial.
  • Not as performant in large datasets (compared to Linked List strategy) as every change in position requires many record updates.

Use this strategy if the order of records will change very infrequently.

Non-Collapsing (Gapped)

Some developers out there will recommend leaving gaps between values in the position column. For example, if the first record is at position 0, then you should place the next record at position 9. In this manner, if you need to insert a new record between the first and second records, then you can insert it with position value = 4, and avoid having to update any other records.

In my opinion, leaving gaps between values is an anti-pattern. In reality, as the gaps start getting filled in, you’ll end up with the Collapsing strategy anyways. The only way to delay the inevitable is to create larger gaps. You, the developer, not only have to write the same code as the Collapsing strategy, but in addition you have to write additional logic for creating and managing gaps as values change.

• Pros:
  • Record updates become cheap since you don’t necessarily need to update other records each time.
• Cons:
  • More development effort.
  • Gaps potentionally require larger datatypes to manage the same amount of records.

Collapsing

Instead of leaving gaps between position values, you can collapse empty space. If you have positions 1, 2, and 3 filled, and the user chooses to delete position 2, then the record in position 3 should change to become position 2.

• Pros:
  • Position values represent what we humans refer to “1st position”, “2nd position” etc. And can be readily presented to the user.
• Cons:
  • Retains the problem of every position change requiring many additional record upates.

Linked List (Previous/Next as a Field)

Best Strategy for Flashcards

Preface

Lettre is a popular Rust crate for setting up a SMTP client. It works well with Gmail’s SMTP server but preparing your Gmail account for use with Lettre (or a lot of other software…) is not immediately clear. This post will show you how to allow ‘Less Secure Apps’ for your google account and how to use Lettre to send emails through smtp.gmail.com.

Setting up Gmail

Allowing ‘Less Secure Apps’

Normally, Google wants you to perform all authentication to apps using Oauth2. However, to reduce the barrier of entry for the developer, we can tell Google to allow ‘Less Secure Apps’ to access our account. Once allowed, we can use basic smtp authentication to authenticate our app for sending emails.

For Non-Managed Users

1. Log in to Gmail with the account you want use for smtp.
2. Access ‘Google Account’ by clicking your avatar at the top-right of the page.
3. From the menu on the left, access ‘Security’ settings.
4. Scroll down to the ‘Less secure app access’ section.
5. Click ‘Turn on access’ and move the slider to the enabled position.

Working with Two-Factor-Auth

If your account has Two-Factor-Auth enabled, then you need to create a new ‘App Password’ to use. An App Password is a random password that can be used by apps to log in without requiring 2FA.

1. Access your ‘Google Account’ ‘Security’ settings like in the steps above.
2. Scroll down to the ‘Signing in to Google’ section.
3. Click on ‘App passwords’ and verify your current credentials.
4. Generate a new password for app ‘Mail’ and device ‘Other’.
5. Save the randomly generated app password, and use it instead of your normal password for authenticating with Lettre.

For Managed Users

If your account is managed by a Google domain account (business, enterprise, etc), then the domain administrator must enable access to Less Secure Apps.

1. Log into the Google Admin Console.
2. Click the ‘Security’ icon.
3. Click the ‘Basic settings’ item.
4. Under the ‘Less secure apps’ section, click ‘Go to settings for less secure apps >>’.
5. Select either ‘Allow users to manage their access to less secure apps’ or ‘Enforce access to less secure apps for all users (Not Recommended).’
6. Save.
7. Now perform the “For Non-Managed Users” steps given above.

Lettre Example

https://github.com/WebeWizard/lettre-gmail-example

Cargo.toml

1
2
3
4
5
6
7
8
9
10
11

[package]
name = "lettre-gmail-example"
version = "0.1.0"
authors = ["webewizard"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
lettre = "0.9.2"
lettre_email = "0.9.2"

main.rs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

extern crate lettre;
extern crate lettre_email;

use lettre::smtp::authentication::IntoCredentials;
use lettre::{SmtpClient, Transport};
use lettre_email::EmailBuilder;

fn main() {
    let smtp_address = "smtp.gmail.com";
    let username = "j.halper@dunmiff.com";
    let password = "Sup3rDup3rP@ssw0rd";

    let email = EmailBuilder::new()
        .to("d.schrute@dunmiff.com")
        .from(username)
        .subject("Which bear is best?")
        .text("Bears eat beets. Bears. Beets. Battlestar Galactica.")
        .build()
        .unwrap()
        .into();

    let credentials = (username, password).into_credentials();

    let mut client = SmtpClient::new_simple(smtp_address)
        .unwrap()
        .credentials(credentials)
        .transport();

    let _result = client.send(email);
}

Serde, BSON, and MongoDB

This purpose of this guide is to give clear examples of inserting and fetching Rust data structures using a MongoDB database.

Vocab

Serde - Rust Library for Serializing and Deserializing data structures.
Serializing - Translating data structures into a format suitable for network transfer or conversion.
Deserializing - Constructing a data structure from a serialized format.
BSON - Binary JSON, serialization format used by MongoDB to represent data structures.
Document - Represents a single data structure.

Cargo.toml

Include the following crates in your cargo.toml file:

1
2
3
4
5

[dependencies]
serde = "1.0"         # serialization library
serde_derive = "1.0"  # automatically derives Serialize and Deserialize impls for your data
bson = "0.9.0"        # BSON library - makes use of Serde to Serialize and Deserialize with BSON
mongodb = "0.3.2"     # Official prototype MongoDB rust driver

Import the crates into your main.rs or lib.rs

1
2
3
4
5

#[macro use(bson, doc)]
extern crate bson;
extern crate mongodb;
extern crate serde;
#[macro use] extern crate serde_derive;    // #[macro use] extends the macros in the crate to our modules

Preparing your data

Serde_derive is capable of implementing Serialize and Deserialize for most simple data structures.
Advanced Serde topics can be found here: Serde Docs

Use #[derive(Serialize, Deserialize)] to automatically implement Serialize and Deserialize for your data.

1
2
3
4
5
6

#[derive(Serialize, Debug, Debug)]
pub struct Person {
  first: String,
  last: String,
  age: u8
}

Preparing a MongoDB connection

In this example we assume MongoDB is installed on the local machine and contains a database called “Test”. We will be using a collection “People” to store our Person data structures
MongoDB rust driver docs are here: MongoDB driver Docs

1
2
3
4

const MONGO_URL: &'static str = "localhost";
const MONGO_PORT: u16 = 27017;
const MONGO_DB: &'static str = "Test";
const MONGO_COLL_PEOPLE: &'static str = "People";

1
2
3
4
5
6

use mongodb::{Client, ThreadedClient};
use mongodb::db::ThreadedDatabase;

let client = Client::connect(MONGO_URL, MONGO_PORT)
    .expect("failed to initialize client");
let people_coll = client.db(MONGO_DB).collection(MONGO_COLL_PEOPLE);

note: MongoDB’s Client is an Arc type backed by a connection pool. You can clone it as many times as you’d like.

Inserting your data into MongoDB (Serializing)

The code below contains lots of matches that can be simplified, but you will be able to see where errors could occur.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

use bson;

// only Serialize is needed here
#[derive(Serialize, Deserialize,)]
pub struct Person {
    pub first: String,
    pub last: String,
    pub age: u16
}

impl Person {

  // takes a borrowed model and returns a simple true/false result
  pub fn insert(&self, model: &T) -> Result<bool,bool> {
      match bson::to_bson(model) {
          Ok(model_bson) => {
              match model_bson{
                  bson::Bson::Document(model_doc) => {
                      match self.coll.insert_one( model_doc, None) {
                          Ok(db_result) => {
                              return Ok(true)
                          },
                          Err(err) => {
                              println!("Failed to insert new model doc into database:\n{}",err);
                              return Err(false)
                          }
                      }
                  },
                  _ => {
                      println!("Failed to create document from new model bson");
                      return Err(false)
                  }
              }
          },
          Err(err) => {
              println!("Failed to create bson from new model:\n{}",err);
              return Err(false)
          }
      }
  }

}

Fetching your data from MongoDB (Deserializing)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

use bson;

// only Deserialize is needed here
#[derive(Serialize, Deserialize,)]
pub struct Person {
    pub first: String,
    pub last: String,
    pub age: u16
}

impl Person {

  // takes a borrowed partial model and returns the model if it matches or None if it doesn't
  // returns a bool instead of error types
  pub fn find_one(&self, model: &T) -> Result<Option<T>, bool> {
      match bson::to_bson(model) {
          Ok(model_bson) => {
              match model_bson{
                  bson::Bson::Document(model_doc) => {
                      match self.coll.find_one( Some(model_doc), None) {
                          Ok(db_result) => {
                              match db_result {
                                  Some(result_doc) => {
                                      match bson::from_bson(bson::Bson::Document(result_doc)) {
                                          Ok(result_model) => {
                                              return Ok(Some(result_model))
                                          },
                                          Err(err) => {
                                              println!("failed to get model from bson");
                                              return Err(false)
                                          }
                                      }
                                  },
                                  None => {
                                      println!("No model found");
                                      return Ok(None)
                                  }
                              }
                          },
                          Err(err) => {
                              println!("Failed to delete doc from database:\n{}",err);
                              return Err(false)
                          }
                      }
                  },
                  _ => {
                      println!("Failed to create document from new model bson");
                      return Err(false)
                  }
              }
          },
          Err(err) => {
              println!("Failed to create bson from model:\n{}",err);
              return Err(false)
          }
      }
  }

}

Only winners get the chicken; and the chicken is really fast