Graph-Algorithm

TL;DR Graph algorithms are memory-bound, not CPU-bound: a 5 billion node, 50 billion edge graph needs 500+ GB just for adjacency, scores, and working memory - it simply does not fit on one machine Four scaling strategies cover most production cases: distributed processing (vertex-cut or edge-cut partitioning), approximate algorithms (sampling, sketching, early stopping), incremental/streaming approaches, and storage-level optimisation (columnar layout, compression, caching) The hidden costs dominate at scale: communication overhead, synchronisation barriers, debugging distributed state, and the operational burden of keeping it all running Full-graph recompute is not feasible at billions of nodes - incremental algorithms and clever approximations are the norm, not the exception Managed services like Neptune Analytics remove much of the partitioning and operations work, at the price of less control Graph algorithms work great on your laptop. PageRank on a 100,000-node graph finishes in seconds. Louvain finds communities instantly. ...

TL;DR A community is a group of nodes with more internal connections than you would expect by chance; community detection finds these groups purely from network structure, not features Modularity is the metric everything uses - 0.3-0.7 is typical for real networks, and 0.9+ usually means you have over-clustered Louvain is the industry standard: fast, greedy, no hyperparameters to tune, and it produces high-modularity divisions; label propagation, spectral clustering, and Girvan-Newman fill the gaps The same technique powers friend-group detection, product clustering, fraud-ring identification, and finding functional modules in biological networks Overlapping communities remain the genuinely hard problem - most standard algorithms assign each node to exactly one group If you’ve ever looked at a social network and wondered “why is this group of people more connected to each other than to the rest of the network?”, you’ve just articulated the community detection problem. ...

TL;DR Graph queries answer relationship questions (paths, positions, influence) that are impossibly expensive in a relational database - the mental shift is from filtering rows to traversing paths PageRank measures a node’s importance by the importance of the nodes pointing to it; it is recursive, converges after 20-30 iterations, and applies far beyond web pages - citations, social influence, recommendations Centrality is an umbrella term: degree (how connected), betweenness (how many paths run through you), closeness (how near everything you are), and eigenvector (whether you are connected to important nodes) each answer a different question Picking the right centrality measure for the question matters more than tuning any single algorithm Managed engines like Neptune Analytics run these at scale without you implementing the iteration loops yourself Graph algorithms often get treated as academic curiosities - something you learn in a computer science course and then never think about again. But they’re actually the hidden backbone of some of the most profitable systems on the internet. ...