Free Code Camp and Learning Node: Uppercaserer

We’re almost done with learnyounode! This was another one that looks harder than it is.  Here we go!

First, as always, the directions and hints:

HTTP UPPERCASERER (Exercise 12 of 13)
  Write an HTTP server that receives only POST requests and converts incoming POST body characters to upper-case and returns it to the client.
  Your server should listen on the port provided by the first argument to your program.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  While you’re not restricted to using the streaming capabilities of the request and response objects, it will be much easier if you do.
  There are a number of different packages in npm that you can use to “transform” stream data as it’s passing through. For this exercise the through2-map package offers the simplest API.
  through2-map allows you to create a transform stream using only a single function that takes a chunk of data and returns a chunk of data. It’s designed to work much like Array#map() but for streams:
  var map = require(‘through2-map’)
     inStream.pipe(map(function (chunk) {
       return chunk.toString().split(”).reverse().join(”)
     })).pipe(outStream)
  In the above example, the incoming data from inStream is converted to a String (if it isn’t already), the characters are reversed and the result is passed through to outStream. So we’ve made a chunk character reverser! Remember though that the chunk size is determined up-stream and you have little control over it for incoming data.
  To install through2-map type:
     $ npm install through2-map
  If you don’t have an Internet connection, simply make a node_modules directory and copy the entire directory for the module you want to use from inside the learnyounode installation directory:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_modules/through2-map
  Documentation for through2-map has been installed along with learnyounode on your system and you can read them by pointing your browser here:

  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/docs/through2-map.html

Okay, first order of business is to install the package.  So, at the $ prompt we type, as instructed, npm install through2-map.  We get this back:

npm WARN package.json chat-example@0.0.0 No repository field.
npm WARN package.json chat-example@0.0.0 No license field.
through2-map@2.0.0 node_modules/through2-map
├── xtend@4.0.1

└── through2@2.0.1 (readable-stream@2.0.6)

Next, we know we’re going to need to require it in the file.  So we create a new file, and this goes at the top:

</div>
<div>var map = require('through2-map');</div>
<div>

We are also going to need an HTTP server, so we add the require statement for that and the skeleton they gave us from before:

</div>
<div>var map = require('through2-map');</div>
<div>var http = require('http');</div>
<div></div>
<div>  var server = http.createServer(function (req, res) {</div>
<div>  // request handling logic...</div>
<div>  })</div>
<div>  server.listen(8000);</div>
<div>

The port is supplied as a command line variable again, so we assign it to a variable:

</div>
<div>var map = require('through2-map');</div>
<div>var http = require('http');</div>
<div>var port = process.argv[2];</div>
<div></div>
<div>  var server = http.createServer(function (req, res) {</div>
<div>  // request handling logic...</div>
<div>  })</div>
<div>  server.listen(port);</div>
<div>

Now for the request handling logic.  They gave us the skeleton above:

</div>
<div>inStream.pipe(map(function (chunk) {</div>
<div>       return chunk.toString().split('').reverse().join('')</div>
<div>     })).pipe(outStream)</div>
<div>

All we need to do is figure out what the inStream and outStream are, and change it from reversing the string to converting it to uppercase.  The streams are actually simple; note that in the hints they say “While you’re not restricted to using the streaming capabilities of the request and response objects, it will be much easier if you do.”  I took this to mean that request and response are our stream objects.  As far as the string conversion, JavaScript has a nice string method called toUpperCase(), and so now our request handling logic looks like this:

</div>
<div>req.pipe(map(function (chunk) {</div>
<div>       return chunk.toString().toUpperCase();</div>
<div>     })).pipe(res);</div>
<div>

We run it, and lo and behold, we passed!  Here’s the final program:

</div>
<div>var map = require('through2-map');</div>
<div>var http = require('http');</div>
<div>var port = process.argv[2];</div>
<div></div>
<div>var server = http.createServer(function (req, res) {</div>
<div></div>
<div>    req.pipe(map(function (chunk) {</div>
<div>       return chunk.toString().toUpperCase();</div>
<div>     })).pipe(res);</div>
<div>       // request handling logic...</div>
<div>     })</div>
<div>server.listen(port);</div>
<div>

All right!  One last exercise and we are done with learnyounode.  See you next time!

FreeCodeCamp and Learning Node: HTTP File Server

Last time we learned how to return a time in response to a request.  Now we’re going to learn to return a file in response to a request.  We’re coming towards the end of the Node tutorials and so things are getting more interesting and complex.  This exercise was an interesting one; we’ll be serving files as part of the challenges for FreeCodeCamp in the future…

For now though, let’s look at  the Directions and Hints:

 ## HTTP FILE SERVER (Exercise 11 of 13)
  Write an HTTP server that serves the same text file for each request it receives.
  Your server should listen on the port provided by the first argument to your program.
  You will be provided with the location of the file to serve as the second command-line argument. You must use the fs.createReadStream() method to stream the file contents to the response.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  Because we need to create an HTTP server for this exercise rather than a generic TCP server, we should use the http module from Node core. Like the net module, http also has a method named http.createServer() but this one creates a server that can talk HTTP.
  http.createServer() takes a callback that is called once for each connection received by your server. The callback function has the signature:
     function callback (request, response) { /* … */ }
  Where the two arguments are objects representing the HTTP request and the corresponding response for this request. request is used to fetch properties, such as the header and query-string from the request while response is for sending data to the client, both headers and body.
  Both request and response are also Node streams! Which means that you can use the streaming abstractions to send and receive data if they suit your use-case.
  http.createServer() also returns an instance of your server. You must call server.listen(portNumber) to start listening on a particular port.
  A typical Node HTTP server looks like this:
     var http = require(‘http’)
     var server = http.createServer(function (req, res) {
       // request handling logic…
     })
     server.listen(8000)
  Documentation on the http module can be found by pointing your browser here:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/http.html

  The fs core module also has some streaming APIs for files. You will need to use the fs.createReadStream() method to create a stream representing the file you are given as a command-line argument. The method returns a stream object which you can use src.pipe(dst) to pipe the data from the src stream to the dst stream. In this way you can connect a filesystem stream with an HTTP response stream.

I admit that at first the directions and hints were a little cryptic, but this exercise turned out to be much easier than I thought.

First, scanning through the material above, we’re going to need http and fs, so we write the require statements for those.  Then, we note that it states that the port and the file path will be provided as command line arguments in our old friend process.argv.  So, as in the past many exercises, we assign those to variables.  That wraps up assignments, and also the easy part.

Next, they give us the skeleton of the server:

</div>
<div> var server = http.createServer(function (req, res) {</div>
<div>       // request handling logic...</div>
<div>     })</div>
<div>     server.listen(8000)</div>
<div>

8000 will of course be replaced by our port, and so our program now looks like this so far:

</div>
<div>var http = require('http');</div>
<div>var fs = require('fs');</div>
<div>var port = process.argv[2];</div>
<div>var filePath = process.argv[3];</div>
<div></div>
<div>var server = http.createServer(function (req, res) {</div>
<div>       // request handling logic...</div>
<div>})</div>
<div>server.listen(port);</div>
<div>

Now we need to write the “request handling logic” they have in there.  We know, from the directions and hints, that we need fs.createReadStream(), and that we’ll also need a src.pipe(dst).  The info on createReadStream is here:

https://nodejs.org/dist/latest-v6.x/docs/api/fs.html#fs_fs_createreadstream_path_options

So, what goes in the parentheses (the parameters) is the file path, which we have a variable for called filePath.  That’s the SOURCE, so we’ll assign it to a variable called src, thus:

</div>
<div>var src = fs.createReadStream(filePath);</div>
<div>

Now, finally, we have src.pipe(dst).  src is the SOURCE (see what I did there?) and dst is the stream we are sending it to.  Here’s where it gets a little tricky.  What are we sending it to?  Well, in the hints we are told, “Both request and response are also Node streams!”  Ahhh, so we want to use the response stream!  So:

</div>
<div>src.pipe(res);</div>
<div>

And…..we pass!  Here’s the final program:

</div>
<div>var http = require('http');</div>
<div>var fs = require('fs');</div>
<div>var port = process.argv[2];</div>
<div>var filePath = process.argv[3];</div>
<div></div>
<div>var server = http.createServer(function (req, res) {</div>
<div>       // request handling logic...</div>
<div>       var src = fs.createReadStream(filePath);</div>
<div>       src.pipe(res);</div>
<div></div>
<div></div>
<div>});</div>
<div>server.listen(port);</div>
<div>

See you next time!

FreeCodeCamp and Learning Node: Time Server

So our next challenge is to build a Time Server.  This one wasn’t too bad.  The only tricky part for me was using the outside library, namely the formatting of the date.

Here are the Directions and Hints:

TIME SERVER (Exercise 10 of 13)
  Write a TCP time server!
  Your server should listen to TCP connections on the port provided by the first argument to your program. For each connection you must write the current date & 24 hour time in the format:
     “YYYY-MM-DD hh:mm”
  followed by a newline character. Month, day, hour and minute must be zero-filled to 2 integers. For example:
     “2013-07-06 17:42”
  After sending the string, close the connection.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  For this exercise we’ll be creating a raw TCP server. There’s no HTTP involved here so we need to use the net module from Node core which has all the basic networking functions.
  The net module has a method named net.createServer() that takes a function. The function that you need to pass to net.createServer() is a connection listener that is called more than once. Every connection received by your server triggers another call to the listener. The listener function has the signature:
     function listener(socket) { /* … */ }
  net.createServer() also returns an instance of your server. You must call server.listen(portNumber) to start listening on a particular port.
  A typical Node TCP server looks like this:
     var net = require(‘net’)
     var server = net.createServer(function (socket) {
       // socket handling logic
     })
     server.listen(8000)
  Remember to use the port number supplied to you as the first command-line argument.
  The socket object contains a lot of meta-data regarding the connection, but it is also a Node duplex Stream, in that it can be both read from, and written to. For this exercise we only need to write data and then close the socket.
  Use socket.write(data) to write data to the socket and socket.end() to close the socket. Alternatively, the .end() method also takes a data object so you can simplify to just: socket.end(data).
  Documentation on the net module can be found by pointing your browser here:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/net.html
  To create the date, you’ll need to create a custom format from a new Date() object. The methods that will be useful are:
     date.getFullYear()
     date.getMonth()     // starts at 0
     date.getDate()      // returns the day of month
     date.getHours()
     date.getMinutes()
  Or, if you want to be adventurous, use the strftime package from npm. The strftime(fmt, date) function takes date formats just like the unix date command. You can read more about strftime at:

  [https://github.com/samsonjs/strftime] ( https://github.com/samsonjs/strftime )

The actual documentation link is:

https://nodejs.org/dist/latest-v6.x/docs/api/net.html

Okay, let’s get rolling. Since I decided to be adventurous, we need to install strftime.  We get:

$ npm install strftime
npm WARN package.json chat-example@0.0.0 No repository field.
npm WARN package.json chat-example@0.0.0 No license field.

strftime@0.9.2 node_modules/strftime

On to the actual program. First, I took the skeleton they give us above:

</pre>
</div>
<div>var net = require('net')</div>
<div>     var server = net.createServer(function (socket) {</div>
<div>       // socket handling logic</div>
<div>     })</div>
<div>     server.listen(8000)</div>
<div>

Now, 8000 isn’t going to necessarily be the socket we want.  We need to use the argument supplied to us.  As always, that’s the third element in process.argv (even though it says first in the hints – I think they mean it’s the first usable argument).  I added that and ran it and the terminal hung up; I figured out that it was still listening because I never told it to stop.  So, I added socket.end() inside var server (where it says “socket handling logic” was a hint to put it there) and I added a console.log to print out the port number to see if I had a working program so far.

Hey, build a little, test a little.  That was one of the mottoes of the moon program and NASA back in the 60s, and it works for me!

 

Here’s what I had so far:

</pre>
</div>
<div>var net = require('net');</div>
<div>var strftime = require('strftime');</div>
<div>var port = process.argv[2];</div>
<div></div>
<div>var server = net.createServer(function (socket) {</div>
<div>        console.log(port);</div>
<div>       // socket handling logic</div>
<div>       socket.end();</div>
<div>     });</div>
<div>     server.listen(port);</div>
<div>

That gives us the output I was looking for.  Obviously, we want our program to do a little more than that, so let’s have a look at the documentation for strftime using the link they gave us.  Note first, of course, that we need to use a require statement.  Next, we see some examples.

</pre>
</div>
<div>console.log(strftime('%B %d, %Y %H:%M:%S')), which returns April 28, 2011 18:21:08</div>
<div></div>
<div>console.log(strftime('%F %T', new Date(1307472705067))), which returns 2011-06-07 18:51:45</div>
<div>

Now, we’re going to need today’s date, so we add a variable for it using the JavaScript Date.now() function.  We want to printout the date and the time, less the seconds.  Returning to the documentation for strftime, we can see that to do that, we can use the first part of the second example, ‘%F’, and most of the last part of the first example, %H:%M. So, now I had:

</pre>
</div>
<div>var net = require('net');</div>
<div>var strftime = require('strftime');</div>
<div>var port = process.argv[2];</div>
<div></div>
<div>var today = Date.now();</div>
<div></div>
<div>var server = net.createServer(function (socket) {</div>
<div>        var data = (strftime('%F %H:%M', new Date(today)));</div>
<div></div>
<div>       // socket handling logic</div>
<div>       socket.end(data);</div>
<div>     });</div>
<div>     server.listen(port);</div>
<div>

This prints the date out correctly, but still doesn’t pass!!!  What???

I got this back:

Your submission results compared to the expected:
                 ACTUAL                                 EXPECTED
────────────────────────────────────────────────────────────────────────────────
   “2016-05-21 16:21”                  ==    “2016-05-21 16:21”
                                                 !=    “”
────────────────────────────────────────────────────────────────────────────────

  ✗  Submission results did not match expected!

Now, what was going on?

Simple: the directions specify that the date must be followed by a newline character.  I added it and lo and behold, my program passed!

</pre>
</div>
<div>var net = require('net');</div>
<div>var strftime = require('strftime');</div>
<div>var port = process.argv[2];</div>
<div></div>
<div>var today = Date.now();</div>
<div></div>
<div>var server = net.createServer(function (socket) {</div>
<div>        var data = (strftime('%F %H:%M', new Date(today))) + '\n';</div>
<div></div>
<div>       // socket handling logic</div>
<div>       socket.end(data);</div>
<div>     });</div>
<div>     server.listen(port);</div>
<div>

And there we go!  See you next time!

FreeCodeCamp and Learning Node: Juggling ASync

On to the next exercise in the FreeCodeCamp learnyounode sequence!  The processes are asynchronous, but we want to tame them so they get us our data in order.  Let’s look at the directions and hints:

JUGGLING ASYNC (Exercise 9 of 13)

  This problem is the same as the previous problem (HTTP COLLECT) in that you need to use http.get(). However, this time you will be provided with three URLs as the first three command-line arguments.
  You must collect the complete content provided to you by each of the URLs and print it to the console (stdout). You don’t need to print out the length, just the data as a String; one line per URL. The catch is that you must print them out in the same order as the URLs are provided to you as command-line arguments.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  Don’t expect these three servers to play nicely! They are not going to give you complete responses in the order you hope, so you can’t naively just print the output as you get it because they will be out of order.
  You will need to queue the results and keep track of how many of the URLs have returned their entire contents. Only once you have them all, you can print the data to the console.
  Counting callbacks is one of the fundamental ways of managing async in Node. Rather than doing it yourself, you may find it more convenient to rely on a third-party library such as [async]( http://npm.im/async ) or [after]( http://npm.im/after ). But for this exercise, try and do it without any external helper library.

Okay, as always, We want to declare our variables first.  We’re going to need http, since we’re using http.get().  We’re also going to want bl, since we are using buffer list for the contents of the URLs, as we did in the previous exercise.  We’re also going to use the skeleton of the call to http.get() that we used in the last exercise:

var bl = require('bl');
var http = require('http');

http.get(addr, function callback(response) {

response.pipe(bl(function (err, data) {
if(err)
console.log(err);
}))

});

Okay, so if we run this now, we get back… a mess.  Remember, there are THREE inputs coming that we want to process one after the other – this suggests a loop.

So, let’s put in a loop:

for(var count = 0; count < process.argv.length; count++) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);

console.log(data.length);
console.log(data.toString());
}));

});
};

Umm…ewww.  No, that was no good.  I messed around with this for awhile, and still didn’t get what I wanted.  At one point, I came close – getting the results back, but not in order.  So, I did some web surfing, as suggested by FreeCodeCamp (you remember how they always say to use Read-Search-Ask, right?), and found this very helpful link on “Four ways of handling asynchronous operations in node.js“.  If you look about halfway down the page, you’ll see a section titled “Reading the files sequentially using callbacks”.

 

Now, here the poster is reading the contents of files, and not URLs.  He is also using fs.readdir.  But the idea is the same.

So, we create a function that contains our http.get() called function readURL().  We need to make it recursive, so it goes through the items provided to us by the arguments.  Keep in mind that, as always with Node, we start with the third element (element[2] since the array starts at 0).  So count’s initial value is 2.

In the response.pipe() statement, we will print the data out – as a string, as mentioned in the directions – and we add that statement.  Also, readURL() is recursive, so if it is successful, we want to call it again, only incrementing count by 1 so that it calls the next argument in process.argv.  If the next argument doesn’t exist, the process terminates and we’re finished.  So, now we have:

var bl = require('bl');
var http = require('http');
var count = 2;

function readURL(count) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);
} else {
console.log(data.toString());
readURL(count + 1);
}
}));

});
}

Let’s run it.

I bet you’re not surprised it doesn’t work, right?  Do you know why?

Because we created a function called readURL()…but we never called it.  We need to add that to the VERY bottom and try again.

 

It worked! And now we can move on to the next exercise.  Here’s the final program:

var bl = require('bl');
var http = require('http');
var count = 2;

function readURL(count) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);
} else {
console.log(data.toString());
readURL(count + 1);
}
}));

});
}

readURL(count);

See you next time!

FreeCodeCamp and Learning Node: HTTP Collect

We’ve completed part 1 of the challenges for FreeCodeCamp’s Node section, and now we’re on to Part 2.  FreeCodeCamp broke this up into sections since the Node tutorial is pretty long.  Last time we learned about using http and the get method, and this lesson builds on that.  As always, you should try it first and then look at what I did. Let’s get right to it!

Here are the directions and hints:

HTTP COLLECT (Exercise 8 of 13)
  Write a program that performs an HTTP GET request to a URL provided to you as the first command-line argument. Collect all data from the server (not just the first “data” event) and then write two lines to the console (stdout).
  The first line you write should just be an integer representing the number of characters received from the server. The second line should contain the complete String of characters sent by the server.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  There are two approaches you can take to this problem:
  1) Collect data across multiple “data” events and append the results together prior to printing the output. Use the “end” event to determine when the stream is finished and you can write the output.
  2) Use a third-party package to abstract the difficulties involved in collecting an entire stream of data. Two different packages provide a
  useful API for solving this problem (there are likely more!): bl (Buffer List) and concat-stream; take your pick!
  To install a Node package, use the Node Package Manager npm. Simply type:
     $ npm install bl
  And it will download and install the latest version of the package into a subdirectory named node_modules. Any package in this subdirectory under your main program file can be loaded with the require syntax without being prefixed by ‘./’:
     var bl = require(‘bl’)
  Node will first look in the core modules and then in the node_modules directory where the package is located.
  If you don’t have an Internet connection, simply make a node_modules directory and copy the entire directory for the package you want to use from inside the learnyounode installation directory:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_modules/bl
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_modules/concat-stream
  Both bl and concat-stream can have a stream piped in to them and they will collect the data for you. Once the stream has ended, a callback will be fired with the data:
     response.pipe(bl(function (err, data) { /* … */ }))
     // or
     response.pipe(concatStream(function (data) { /* … */ }))
  Note that you will probably need to data.toString() to convert from a Buffer.
  Documentation for both of these modules has been installed along with learnyounode on your system and you can read them by pointing your browser here:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/docs/bl.html

  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/docs/concat-stream.html

Okay, let’s break this down step by step.  I didn’t want to have to use a bunch of appends, because that didn’t sound elegant to me.  I went with one of the third party methods instead, and I picked the Buffer List package.  You can always use the other one.

Type “npm install bl first” at the $ prompt (in other words, NOT in your program), and you’ll get:

<YOUR USERNAME>:~/workspace $ npm install bl
npm WARN package.json chat-example@0.0.0 No repository field.
npm WARN package.json chat-example@0.0.0 No license field.
bl@1.1.2 node_modules/bl
└── readable-stream@2.0.6 (string_decoder@0.10.31, process-nextick-args@1.0.7, util-deprecate@1.0.2, inherits@2.0.1, core-util-is@1.0.2, isarray@1.0.0)

Now Buffer List is installed and you can use it!

Time to start the program.  This is going to use the get method of http again, so we can borrow from the last exercise.  We also need to require the bl package as noted in the hints.  I left require fs in there but you don’t need it for this exercise.

var bl = require('bl');
var fs = require('fs');
var http = require('http');
var addr = process.argv[2];
Remember that the third element of process.arg is the path to the data we want.  That takes care of the variables and requires, so let’s move on to the rest of the program.
http.get(addr, function callback(response) {

&amp;nbsp; &amp;nbsp; &amp;nbsp;/* Do stuff here */

}).on('error', (e) =&amp;gt; {
&amp;nbsp; console.log(`Got error: ${e.message}`);
});

The framework of the program hasn’t changed.  What IS going to change is the part where I put “/* Do stuff here */”. I put the line in the hint in there:

response.pipe(bl(function (err, data) { /* ... */ }))&amp;nbsp;

Just for fun I put

console.log(data.length);

in there, and got a warning that it expected an error to be handled.  From one of the previous lessons, I remembered that we add:

if(err)
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; console.log(err);

I ran it then, even though it’s incomplete, and lo and behold I got a length back that matched the expected answer!  Almost there!  Now, the next thing is to print the contents of the buffer, and they tell you how to do that with a little hint that you need to convert it to a string, like so:

console.log(data.toString());

Do that, and the program passes!

 

Just a quick aside:  It actually took me a half hour and several false trails before I got this.  Believe me, I’m making it seem much easier than it was here!

Anyway, here’s the entire program!

var bl = require('bl');
var fs = require('fs');
var http = require('http');
var addr = process.argv[2];

http.get(addr, function callback(response) {

response.pipe(bl(function (err, data) {
if(err)
console.log(err);

console.log(data.length);
console.log(data.toString());

}))


}).on('error', (e) => {
console.log(`Got error: ${e.message}`);
});

FreeCodeCamp and Learning Node: HTTP Client

Welcome back to the Node tutorial series! Now we’re going to actually interact with web requests.  Hang on, things are going to get wild…

As always, TRY THIS YOURSELF FIRST!  DO THE RESEARCH ON THE WEB TO LEARN THINGS, AND NOT JUST BY READING WHAT i WROTE BELOW!!!

We begin with the Directions and Hints that the tutorial gives us:

Write a program that performs an HTTP GET request to a URL provided to you as the first command-line argument. Write the String contents of each “data” event from the response to a new line on the console (stdout).
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  For this exercise you will need to use the http core module.
  Documentation on the http module can be found by pointing your browser here:
  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/http.html
  The http.get() method is a shortcut for simple GET requests, use it to simplify your solution. The first argument to http.get() can be the URL you want to GET; provide a callback as the second argument.
  Unlike other callback functions, this one has the signature:
     function callback (response) { /* … */ }
  Where the response object is a Node Stream object. You can treat Node Streams as objects that emit events. The three events that are of most interest are: “data”, “error” and “end”. You listen to an event like so:
     response.on(“data”, function (data) { /* … */ })
  The “data” event is emitted when a chunk of data is available and can be processed. The size of the chunk depends upon the underlying data source.

  The response object / Stream that you get from http.get() also has a setEncoding() method. If you call this method with “utf8”, the “data” events will emit Strings rather than the standard Node Buffer objects which you have to explicitly convert to Strings.

The actual web page link is:

https://nodejs.org/dist/latest-v6.x/docs/api/http.html

Okay, first thing is to declare our variables.  we’ll need a require statement for http – note that it has to be in lowercase letters.  We will also assign the address they give us on the command line to addr.  As you know, it’s the third argument but element[2] due to the fact that arrays begin with 0!  We have:

var http = require('http');
var addr = process.argv[2];

So, we’re going to call http.get(); it’s arguments will be the URL they give us as well as a callback function.  Inside that call to get we will call the response.on function. We can also add a setEncoding statement as noted in the hints.  So:

var http = require('http');
var addr = process.argv[2];

http.get(addr, function callback(response) {
response.setEncoding('utf8');
response.on("data", function (data) { /* ... */ });
// consume response body

})

We need to handle an error.  A little web searching gives us a chained method:

var http = require('http');
var addr = process.argv[2];

http.get(addr, function callback(response) {
response.setEncoding('utf8');
response.on("data", function (data) {  /* ... */ });
// consume response body

}).on('error', (e) => {
console.log(`Got error: ${e.message}`);
});

Okay, we’ve done the easy parts.  Now, We need to figure out what goes in the response.on function.

Well, we want to write the results to the console, right?  No problem!  So,

var http = require('http');
var addr = process.argv[2];

http.get(addr, function callback(response) {
response.setEncoding('utf8');
response.on("data", function (data) { console.log(data); });
// consume response body

}).on('error', (e) => {
console.log(`Got error: ${e.message}`);
});

Ouch!  Errors everywhere!

Okay, a little trial and error, and a little reading of the http.get() documentation, reveals that first, we need to add a response.resume() statement to keep things moving.  Second, we want the data from addr to be printed out.  So we add the resume statement and modify the response.on() statement per these items, and our final program is:

PROGRAM (program6.js):

var http = require('http');
var addr = process.argv[2];

http.get(addr, function callback(response) {
response.setEncoding('utf8');
response.on("data", function (addr) { console.log(addr); });
// consume response body
response.resume();
}).on('error', (e) => {
console.log(`Got error: ${e.message}`);
});

And there you have it!  See you next time!

FreeCodeCamp and Learning Node: Make It Modular

This one drove me nuts, I’ll admit it.  The directions were sparse, but eventually I figured it out.  You can’t just copy-paste from the previous exercise where we filtered I/O, though at first it seems like you can.  Turns out there’s a lot of extras thrown into this one, such as error checking.

Okay, let’s look at the Instructions and hints:

 

MAKE IT MODULAR (Exercise 6 of 13)

This problem is the same as the previous but introduces the concept of modules. You will need to create two files to solve this.

Create a program that prints a list of files in a given directory, filtered by the extension of the files. The first argument is the directory name and the second argument is the extension filter. Print the list of files (one file per line) to the console. You must use asynchronous I/O.

You must write a module file to do most of the work. The module must export a single function that takes three arguments: the directory name, the filename extension string and a callback function, in that order. The filename extension argument must be the same as what was passed to your program. Don’t turn it into a RegExp or prefix with “.” or do anything except pass it to your module where you can do what you need to make your filter work.

 

The callback function must be called using the idiomatic node(err, data) convention. This convention stipulates that unless there’s an error, the first argument passed to the callback will be null, and the second will be your data. In this exercise, the data will be your filtered list of files, as an Array. If you receive an error, e.g. from your call to fs.readdir(), the callback must be called with the error, and only the error, as the first argument.

You must not print directly to the console from your module file, only from your original program.

In the case of an error bubbling up to your original program file, simply check for it and print an informative message to the console.

 

These four things are the contract that your module must follow.

   » Export a single function that takes exactly the arguments described.
   » Call the callback exactly once with an error or some data as described.
   » Don’t change anything else, like global variables or stdout.

» Handle all the errors that may occur and pass them to the callback.

The benefit of having a contract is that your module can be used by anyone who expects this contract. So your module could be used by anyone else who does learnyounode, or the verifier, and just work.

 ─────────────────────────────────────────────────────────────────────────────

## HINTS

Create a new module by creating a new file that just contains your directory reading and filtering function. To define a single function export, you assign your function to the module.exports object, overwriting what is already there:

 

module.exports = function (args) { /* … */ }

 

Or you can use a named function and assign the name.

To use your new module in your original program file, use the require() call in the same way that you require(‘fs’) to load the fs module. The only difference is that for local modules must be prefixed with ‘./’. So, if your file is named mymodule.js then:

var mymodule = require(‘./mymodule.js’)

 

The ‘.js’ is optional here and you will often see it omitted.

 

You now have the module.exports object in your module assigned to the mymodule variable. Since you are exporting a single function, mymodule is a function you can call!

Also keep in mind that it is idiomatic to check for errors and do early-returns within callback functions:

     function bar (callback) {
       foo(function (err, data) {
         if (err)
           return callback(err) // early return
         // … no error, continue doing cool things with `data`
         // all went well, call callback with `null` for the error argument
         callback(null, data)
       })

}

Let’s tackle the module first.  Basically, by module, we mean a separate file.  The hints give you module.exports = function (args) { /* … */ }, except it turns out that’s not as helpful as it looks.

 

First thing to realize is that since you’re going to be calling the fs.readdir and path packages, you need them to be required at the beginning of the file.  That’s the easy part!

Next, we add the line given above.  Now, you see the part in parentheses that says args?  Yeah, well, the directions state that you need to have the module export a function that takes THREE arguments and that is very clearly only ONE.  So, what would those arguments be? The directions tell us: “the directory name, the filename extension string and a callback function, in that order.”

 

So, let’s add those:

var fs = require('fs');
var path = require('path');
module.exports = function (filename, extsn, callback) {

});

 

So far, so good.  But, what do we put inside that function?

First, the directions tell us not to modify the extension in the main program, and we know from the last exercise that we’ll need a dot in front of it when we run the path.extname() method.  So let’s add a dot:

extsn = "." + extsn;

Next, we need to read the directory.  We did this in the last assignment, so we’ll just copy that over:

fs.readdir(filename, function callback(err, list) {
 /* do stuff here */
});

We need to handle any errors that pop up, so we’ll add the error catching routine the specify above:

fs.readdir(filename, function callback(err, list) {
 if(err) {
 return callback(err);
 }
 /* do stuff here */
});

If there’s no error, we want to run the filtering routine.  I had the routine add the files that matched extensions to a new array, called ansArray (Answer Array) and the function returns that.  The filter is similar to what we did before, using a loop and comparing the extension retrieved by path.extname() to the extsn variable we passed to the function.  If they match, we push the item into the array.  So, we have:

fs.readdir(filename, function callback(err, list) {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if(err) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;return callback(err);
&nbsp; &nbsp; &nbsp;}&nbsp;else {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;for (var item = 0; item &lt; list.length; item++) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if (path.extname(list[item]) === extsn) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ansArray.push(list[item]);
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;}
&nbsp; &nbsp; &nbsp;/* do stuff here */
});

To return ansArray, we use the other line they give us in the hints – callback(null, data):

return&nbsp;callback(null, ansArray);

That’s it.  Here’s the complete mymodule.js :

MODULE (mymodule.js):
var fs = require('fs');
var path = require('path');
var ansArray = [];

module.exports = function (filename, extsn, callback) {
extsn = "." + extsn;
fs.readdir(filename, function (err, list) {
if(err) {
return callback(err);
} else {
for (var item = 0; item < list.length; item++) {
if (path.extname(list[item]) === extsn) {
ansArray.push(list[item]);
}
}
return callback(null, ansArray);
}
});
};

After all that, the main program itself is relatively simple.  First, we assign the directory path (element[2] of process.argv) to a variable called filename (because I never got around to changing it from the previous exercise), and then we assign the extension for which we are hunting (element[3] of process.argv) to a variable called extsn.  Finally, as noted in the hints, we require mymodule.js.

Next, we call mymodule, which is a single function, and we call it with three arguments, as specified.  We send it the directory path, the extension on which to filter, and a callback function.  We handle any error it passes back to us.  Then we print out the results using a loop and iterating through list.  Here’s the program:

 

PROGRAM (program5.js)

var filename = process.argv[2];
var extsn = process.argv[3];
var mymodule = require('./mymodule.js');

mymodule(filename, extsn, function(err, list) {
if (err) {
console.log(err);
}
for (var j = 0; j < list.length; j++) {
console.log(list[j]);
}
});

And there we go!  See you next time!

FreeCodeCamp and Learning Node: Filtered LS

This one is similar to the previous two exercises except that instead of a file, we’re reading a directory’s contents.  So, we can use those as a jumping-off point.  The filter itself is simple as well – I didn’t call an outside function like I would with a program in the browser.  Hold that thought, though….we’ll see it again later!

As usual, if you haven’t at least attempted this one, GO TRY IT FIRST!  The description below makes it seem like I put this together easily and quickly, but truthfully, it took me awhile to get it, and there were a few false starts and rabbit trails.  So don’t get discouraged if you don’t get it right away!

For now, let’s take a look at the directions and hints for this exercise:

FILTERED LS (Exercise 5 of 13)

  Create a program that prints a list of files in a given directory, filtered by the extension of the files. You will be provided a directory name as the first argument to your program (e.g. ‘/path/to/dir/’) and a file extension to filter by as the second argument.
  For example, if you get ‘txt’ as the second argument then you will need to filter the list to only files that end with .txt. Note that the second argument will not come prefixed with a ‘.’.
  Keep in mind that the first arguments of your program are not the first values of the process.argv array, as the first two values are reserved for system info by Node.
  The list of files should be printed to the console, one file per line. You must use asynchronous I/O.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  The fs.readdir() method takes a pathname as its first argument and a callback as its second. The callback signature is:
function callback (err, list) { /* ... */ } &amp;amp;amp;nbsp;

where list is an array of filename strings.

Documentation on the fs module can be found by pointing your browser here:

file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/fs.html

You may also find node’s path module helpful, particularly the extname method.

Documentation on the path module can be found by pointing your browser here:

file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/path.html

The documentation link, as I’ve mentioned before, is

https://nodejs.org/dist/latest-v6.x/docs/api/

So, we’re going to start this one the same as last time, with the require statement for fs.  Then, they got a little sneaky.  If you actually go to the documentation page for path, you’ll see that it needs its OWN require statement.  They don’t tell you that, so get in the habit of checking the documentation!

Next, process.argv[2] is … well, actually, here it’s a directory path, so I should have named it dirPath or something.  Anyway, I copied it from before.  They note above that element[3] in the arguments is the extension we’re filtering for, so I assigned that (process.argv[3]) to the variable extsn.  They also note that the extension will come without a “.” added to it, so I added that as well.  So far we have:

var fs = require('fs');
var path = require('path');
var filename = process.argv[2];
var extsn = "." + process.argv[3];

Okay, we’re done declaring variables.  On to the meat of the program!  We next add the fs.readdir() function with the same arguments we gave the fs.readFile() function before:

var bufFile = fs.readdir(filename, function callback(err, list) {&nbsp;
});

As noted above, filename is really the directory path.

 

We need to make sure there’s no error – if(!err) – and then we need to loop through each of the files in the directory.  It’s not immediately clear, but the files are contained in ‘list’ in the fs.readdir() method above.  So, we loop through list.

path.extname() pulls off the the last three characters of the filepath and the dot (that’s why we needed to add a dot to extsn).  So, as we loop through each item in list, we’ll compare the item’s extension as returned by path.extname() to extsn and if they’re the same, we’ll write the file’s name to the console.

for (var item = 0; item &lt; list.length; item++) {

 if (path.extname(list[item]) === extsn) {
 console.log(list[item]);
 }

 

And that completes the program!  Here’s the whole thing, for reference:

var fs = require('fs');
var path = require('path');
var filename = process.argv[2];
var extsn = "." + process.argv[3];

var bufFile = fs.readdir(filename, function callback(err, list) {
if(!err) {
for (var item = 0; item < list.length; item++) {

if (path.extname(list[item]) === extsn) {
console.log(list[item]);
}
}
}
});

See you next time!

FreeCodeCamp and Learning Node: My First Async I/O

Last time we looked at a synchronous program in Node that read a file, split it into lines, and then counted the number of lines using the new line character as a guide.  Today, we’re going to do the same thing, using an asynchronous function, which is a key strength of Node.  If you don’t know what any of this means, well, first, why haven’t you tried this in the Nodeschool tutorial?  Go do that first, I’ll wait.

Next, read on!

Here are the directions and hints.  Once again, remember that the link for the documentation is https://nodejs.org/dist/latest-v6.x/docs/api/:

MY FIRST ASYNC I/O! (Exercise 4 of 13)

  Write a program that uses a single asynchronous filesystem operation to read a file and print the number of newlines it contains to the console (stdout), similar to running cat file | wc -l.

  The full path to the file to read will be provided as the first command-line argument.

 ─────────────────────────────────────────────────────────────────────────────

 # HINTS

  The solution to this problem is almost the same as the previous problem except you must now do it the Node.js way: asynchronous.

  Instead of fs.readFileSync() you will want to use fs.readFile() and instead of using the return value of this method you need to collect the value from a callback function that you pass in as the second argument. To learn more about callbacks, check out:

  [ https://github.com/maxogden/art-of-node#callbacks]

(https://github.com/maxogden/art-of-node#callbacks ).

  Remember that idiomatic Node.js callbacks normally have the signature:

     function callback (err, data) { /* … */ }

  so you can check if an error occurred by checking whether the first argument is truthy. If there is no error, you should have your Buffer object as the second argument. As with readFileSync(), you can supply ‘utf8’ as the second argument and put the callback as the third argument and you will get a String instead of a Buffer.

  Documentation on the fs module can be found by pointing your browser here:

  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/fs.html

Now, it says that we are going to do the same thing as last time, only using an asynchronous method rather than a synchronous one.  That means we can use parts of the previous program we wrote.  The first line and the second line, where we call the file system method and assign the file path to a variable, will be the same.

var fs = require(‘fs’);

var filename = process.argv[2];

Line 3 is going to be wildly different, though.  We are using a different method, an asynchronous one, namely fs.readFile().  For an explanation of how the whole asynchronous thing works, the link they gave in the hints (see above) isn’t bad.  Anyway, the readFile() function takes several arguments.  First is the name and path of the file you want to read.  We saved that in ‘filename’.  Second is ‘utf8’, which means you’ll get the info back as text, similar to how we did in the previous program.  Third is where it gets a little weird.  We call an anonymous function – we’ve done this sort of thing before, in the Front End Development exercises, so it shouldn’t be too unfamiliar.  That function is called a ‘callback’ function (see the link they gave for more), and it takes two arguments.  The first is err, so if something goes wrong it will return an error.  The second is called data here, but you could call it contents or something else if you wanted to.  This will be the actual contents of your file (as opposed to the file name and path we stored in ‘filename’).  So, here’s what we have now:

var fs = require(‘fs’);
var filename = process.argv[2];
var bufFile = fs.readFile(filename, ‘utf8’, function callback(err, data) {

});

Now, two questions:

What do we put into that function?

What about the rest of the program from the previous exercise?

Glad you asked!  First, in the function, we want to convert the file to a string, then split it using the ‘\n’ as the separator.  We did this before, so that answers the second question.

I left out the final line, because I wanted to talk briefly about it, and that’s the line where we print the number of new lines to the console.  That HAS to go inside the function.  Why?  Because if you put it at the end, the console.log() will execute while the file is still being read and split.  You’ll get back undefined.  See, this is the whole thing about how asynchronous ops work.  The program can do other stuff while it’s waiting for time sucking operations to complete, and if you put console.log() outside the function, it will be all, “Hey, I can do this while I’m waiting”.  So, inside the function goes console.log(), and we end up with:

var fs = require(‘fs’);
var filename = process.argv[2];
var bufFile = fs.readFile(filename, ‘utf8’, function callback(err, data) {
var strFile = bufFile.toString();
var cnt = strFile.split(‘\n’);
console.log(cnt.length – 1);

});

And there you have it!  Our first Asynchronous I/O program!

FreeCodeCamp and Learning Node: My First I/O

The first real challenge for the Node tutorial is learning I/O from files.  This one was a little more difficult than Baby Steps.  My solution is probably a little clumsy, but I think it’s easy to follow.  Let’s dive in!

So, here’s the learnyounode instructions for My first I/O.

 ## MY FIRST I/O! (Exercise 3 of 13)

  Write a program that uses a single synchronous filesystem operation to read a file and print the number of newlines (\n) it contains to the console (stdout), similar to running cat file | wc -l.

  The full path to the file to read will be provided as the first command-line argument (i.e., process.argv[2]). You do not need to make your own test file.

 ─────────────────────────────────────────────────────────────────────────────

 ## HINTS

  To perform a filesystem operation you are going to need the fs module from the Node core library. To load this kind of module, or any other “global” module, use the following incantation:

  var fs = require(‘fs’)

  Now you have the full fs module available in a variable named fs.

  All synchronous (or blocking) filesystem methods in the fs module end with ‘Sync’. To read a file, you’ll need to use fs.readFileSync(‘/path/to/file’). This method will return a Buffer object containing the complete contents of the file.

  Documentation on the fs module can be found by pointing your browser here:

  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/fs.html

  Buffer objects are Node’s way of efficiently representing arbitrary arrays of data, whether it be ascii, binary or some other format. Buffer objects can be converted to strings by simply calling the toString() method on them. e.g. var str = buf.toString().

  Documentation on Buffers can be found by pointing your browser here:

  file:///home/ubuntu/.nvm/versions/node/v4.4.3/lib/node_modules/learnyounode/node_apidoc/buffer.html

  If you’re looking for an easy way to count the number of newlines in a string, recall that a JavaScript String can be .split() into an array of substrings and that ‘\n’ can be used as a delimiter. Note that the test file does not have a newline character (‘\n’) at the end of the last line, so using this method you’ll end up with an array that has one more element than the number of newlines.

First of all, those document addresses are useless.  Those addresses are great if you are on the Node server, but in learnyounode, you’ll get nothing.  You can actually go here to look at the cocumentation, and just search for the term you want:

https://nodejs.org/dist/latest-v6.x/docs/api/

 

For this exercise, in the column on the left, it’s File Server (FS) and Buffer that you want.

Now, if it’s not obvious, Node runs on a server.  That’s why all this stuff looks so arcane compared to HTML, CSS, and yes, writing JavaScript in the browser environment.  Node runs off a command line, too, which adds to the alien feel for Front End Developers.  I have experience with Java and this is still taking some getting used to because I did all my Java development in IDEs.  Let’s work this through.

So, the first hint they give, loading fs, does not take place on the command line – it goes in the file you are writing.  Create a new file (I covered this in the last Node Learning Node post).  Name it whatever you want – I called mine program2.js, because I didn’t want to overwrite program.js, which had my answer to the ‘Baby Steps’ tutorial.

Just copy and paste var fs = require(‘fs’), add a semicolon (hey, we’re writing JavaScript, remember?) and that will be the first line of your program.

 

That was the easy part.

 

Okay, learnyounode is GIVING us the file we need to examine – we don’t need to create one.  They are supplying it to us as a command line argument – remember from the last tutorial that the command line arguments object (process.argv) has ‘node’ as the first element, the file path as the second element, and then third (process.argv[2])and up we have whatever information we need.  So, the file information – the location of it, or path – will be process.argv[2].  Let’s assign that to a variable, and our program now looks like:

var fs = require(‘fs’);

var filename = process.argv[2];

Save early, save often.  You remember that from any word processing classes you took, right?

Next, we’ll start using the other commands given in the hints.  First is fs.readFileSync(‘/path/to/file’).  We saved the path to the file in the variable filename.  The command will return the contents of the file as a Buffer object, so we’re going to save that in another new variable we’ll call bufFile (Buffer File -I’m so original).  Here’s what our file looks like now:

var fs = require(‘fs’);
var filename = process.argv[2];

var bufFile = fs.readFileSync(filename);

We need to convert that Buffer object to a string so we can perform string operations on it.  We’ll do that with toString() and assign it to – you guessed it – yet another new variable, which we’ll call strFile, for String File:

var strFile = bufFile.toString();

Believe it or not, we’re almost there.  Using the split() method (I covered it in this post here) with ‘\n’ – for new line – as the separator, we’ll get an array back that will have each line in the file as a separate element.  We’ll save that in one last variable, called cnt (for count):

var cnt = strFile.split(‘\n’);

Now, as noted in the hints, the last line won’t have a new line at the end of it, and we subtract that line from the total since it doesn’t have one.  We can print it to the console now, and that finishes our program, which now looks like this:

var fs = require(‘fs’);

var filename = process.argv[2];

var bufFile = fs.readFileSync(filename);

var strFile = bufFile.toString();

var cnt = strFile.split(‘\n’);

console.log(cnt.length – 1);

If you’ll remember, I called this one program2.js, so I run it as:

learnyounode verify program2.js

And it comes back as passed!

And there you go.  Let me know if there any questions in the comments!