Merge Next into Master #1

Merged
thomascole merged 20 commits from next into master 2023-04-13 12:21:08 -04:00
13 changed files with 130 additions and 32 deletions
Showing only changes of commit 5e4b1f422f - Show all commits

View File

@ -1,3 +1,3 @@
<footer>
<p class="text-center text-sm">&copy; 2022 Thomas Cole</p>
<p class="text-center text-sm">&copy; 2023 Thomas Cole</p>
</footer>

View File

@ -1,3 +0,0 @@
<h1 class="text-5xl font-semibold">
<slot/>
</h1>

View File

@ -1,3 +0,0 @@
<h1 class="text-4xl font-semibold">
<slot/>
</h1>

View File

@ -1,3 +0,0 @@
<h1 class="text-3xl font-semibold">
<slot/>
</h1>

View File

@ -1,6 +1,7 @@
---
import Divider from "../components/misc/Divider.astro";
import Layout from "./Layout.astro";
import '../styles/Markdown.css';
const { frontmatter } = Astro.props;
---

View File

@ -10,7 +10,7 @@ const posts = (await Astro.glob("./blog/*.{md,mdx}"))
new Date(a.frontmatter.date).valueOf()
).slice(0,5);
const projects = (await Astro.glob("./projects/*.{md,mdx}")).sort(
const projects = (await Astro.glob("./projects/**/*.{md,mdx}")).sort(
(a,b) =>
new Date(b.frontmatter.date).valueOf()-
new Date(a.frontmatter.date).valueOf()

View File

@ -4,7 +4,7 @@
import BlogProjectCard from "../../components/misc/BlogProjectCard.astro";
export async function getStaticPaths({ paginate }: { paginate: any }) {
const posts = (await Astro.glob("./*.{md,mdx}")).sort(
const posts = (await Astro.glob("./**/*.{md,mdx}")).sort(
(a, b) =>
new Date(b.frontmatter.date).valueOf() -
new Date(a.frontmatter.date).valueOf()

View File

@ -1,12 +1,12 @@
---
layout: "../../layouts/InteractiveDemoLayout.astro"
layout: "../../../layouts/InteractiveDemoLayout.astro"
title: "Quicksort Demo"
tagline: "A simple visualization of the quicksort algorithm using p5.js"
date: "December 23, 2022"
image: "https://images.unsplash.com/photo-1669399213378-2853e748f217?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=1632&q=80"
imageattr: "https://unsplash.com/photos/5dgXQJ7ezuU"
---
import Quicksort from './quicksort_extra/quicksort.svelte';
import Quicksort from './quicksort.svelte';
<Quicksort client:load/>
---

View File

@ -0,0 +1,86 @@
---
layout: "../../../layouts/InteractiveDemoLayout.astro"
title: "Visual Sitemap"
tagline: "Using Cytoscape.js to visualize all the pages of a website."
date: "January 6, 2023"
image: "https://images.unsplash.com/photo-1639322537228-f710d846310a?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=1632&q=80"
imageattr: "https://unsplash.com/photos/T9rKvI3N0NM"
---
import Sitemapgraph from './sitemapgraph.svelte'
### Background
I was given a project at work to help improve the SEO of our organization's website. We quickly realized that we did not have a great overview of everything on the website. This began the search for a tool to help visualize all the pages on the website. While there are lots of expensive SEO tools we struggled to find something that would just give us a hierarchical view of pages. So if the tool doesn't exist, we need to make the tool.
### Designing the tool
![xkcd](https://imgs.xkcd.com/comics/manuals.png)
I set out to make a generic site mapper that could take a url and crawl all other pages on the site. I wanted it to have a simple command line interface, with the ability to save results for later viewing, and be able to spin up an integrated web server for the visualization. I also took the opportunity to write it all in Typescript, and I mean really use it not just write Javascript in a .ts file.
The command line interface and saving a file to disk are straight forward enough. The node fs module lets us read and write to the host filesystem and [Yargs](http://yargs.js.org/) can handle the command line stuff for us. Even crawling a website wasn't too difficult. The [sitemap-generator](https://www.npmjs.com/package/sitemap-generator) project handles all the heavy lifting. The generator creates a crawler object that keeps track of every page it visits in a queue. By looking at that queue after the crawler finishes we can get json objects of every page visited and importantly for us the page that linked to it. Finally, [Express](https://expressjs.com/) framework will handle the web server bits for the visualization.
### Putting it all together
So we have a great big array of json objects that have a url and a referrer among other things. How do we take this data and turn it into something usable?
***Graphs***
My CS200 professor can rest easy, I was paying attention and did learn something. Side note: for the sake of this app, we made the decision to not care about back links. While this doesn't change the implementation of our graph it does simplify displaying the graph as we don't have to worry about loops. On to the code.
```typescript
class Vertex {
name: string;
ajd: Array<Edge>;
constructor(name: string) {
this.name = name;
this.ajd = new Array<Edge>();
}
toString() {
return this.name;
}
}
class Edge {
destination: Vertex;
constructor(vertex: Vertex) {
this.destination = vertex;
}
}
```
Each Vertex in the graph has a name and an array of edges. Each edge has a destination vertex. Easy enough. Now for the graph. We keep a map of vertex names and its corresponding vertex. If the vertex does not exist in the map we add it to the map and return the vertex. Adding an edge then becomes as simple as passing the name of the two vertices we want to connect. If both vertices don't exist they are created in the graph and we push the new edge to the source vertex adjacency array. Thats really it, there is some code omitted from the example below for printing the graph as well serializing the graph object so it can be saved to disk.
```typescript
class Graph {
vertices: Map<string, Vertex>;
constructor() {
this.vertices = new Map<string, Vertex>();
}
getVertex(name: string) {
let v = this.vertices.get(name);
if (v === undefined) {
v = new Vertex(name);
this.vertices.set(name, v);
}
return v;
}
addEdge(source: string, dest: string) {
const v = this.getVertex(source);
const w = this.getVertex(dest);
v.ajd.push(new Edge(w));
}
}
```
Now that we have a graph representation of our site we just need to convert it something a bit more visually appealing. [Cytoscape.js](https://js.cytoscape.org/) will handle that for us. I was tempted to use something like p5.js or D3 to handle the visualization but Cytoscape seemed better built for graph and network data structures. The translation from our graph data structure to Cytoscape is super easy, just pass in an array of vertices and edges.
Finally after all that crawling and parsing we are left with the desired result. A interactive hierarchical view of every page on a website. Is the tool perfect, no, but it was good enough for what we needed.
<Sitemapgraph client:load/>

View File

@ -0,0 +1,11 @@
<script>
</script>
<style>
</style>
<div>
</div>

View File

@ -1,18 +0,0 @@
---
layout: "../../layouts/InteractiveDemoLayout.astro"
title: "Visual Sitemap"
tagline: "Using d3.js to visualize all the pages of a website."
date: "January 6, 2023"
image: "https://images.unsplash.com/photo-1639322537228-f710d846310a?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=1632&q=80"
imageattr: "https://unsplash.com/photos/T9rKvI3N0NM"
---
This demo was originally written as part of a job application to work for the IT department at my university. It was the first project I wrote with the aid of the p5.js library.
I used this demo as a way to learn how to visualize the quicksort algorithm.\
\
I took inspiration from the YouTube video: *[15 Sorting Algorithms in 6 Minutes](https://www.youtube.com/watch?v=kPRA0W1kECg)*. The source code for those interested can be found on my *[Github](https://github.com/thomaspcole/p5QuicksortVisualization)*.\
\
The demo works by generating an array of 200 values then randomly shuffling the array. We then call the ```quicksort``` function to sort the array. At each iteration of the function, the code takes a 'snapshot' of the data array at that iteration.
By iterating through the 'snapshot' array we get our final animation. It also allows for stepping forward and backwards to see each step of the animation.

27
src/styles/Markdown.css Normal file
View File

@ -0,0 +1,27 @@
.astro-code{
margin-bottom: 1rem;
padding: 8px;
}
h1,h2,h3,h4,h5,h6,p,hr{
margin-bottom: 1rem;
}
h1{
font-size: 3rem;
line-height: 1;
}
h2{
font-size: 2.25rem;
line-height: 2.5rem;
}
h3{
font-size: 1.875rem;
line-height: 2.25rem;
}
img{
margin: auto;
}