LINQ and collections
Most collections model a sequence of elements. You can use LINQ to query any collection type. Other LINQ methods find elements in a collection, compute values from the elements in a collection, or modify the collection or its elements. These examples help you learn about LINQ methods and how you can use them with your collections, or other data sources.
How to find the set difference between two lists
This example shows how to use LINQ to compare two lists of strings and output those lines that are in first collection, but not in the second. The first collection of names is stored in the file names1.txt:
Bankov, Peter
Holm, Michael
Garcia, Hugo
Potra, Cristina
Noriega, Fabricio
Aw, Kam Foo
Beebe, Ann
Toyoshima, Tim
Guy, Wey Yuan
Garcia, Debra
The second collection of names is stored in the file names2.txt. Some names appear in both sequences.
Liu, Jinghao
Bankov, Peter
Holm, Michael
Garcia, Hugo
Beebe, Ann
Gilchrist, Beth
Myrcha, Jacek
Giakoumakis, Leo
McLin, Nkenge
El Yassir, Mehdi
The following code shows how you can use the Enumerable.Except method to find elements in the first list that aren't in the second list:
// Create the IEnumerable data sources.
string[] names1 = File.ReadAllLines("names1.txt");
string[] names2 = File.ReadAllLines("names2.txt");
// Create the query. Note that method syntax must be used here.
var differenceQuery = names1.Except(names2);
// Execute the query.
Console.WriteLine("The following lines are in names1.txt but not names2.txt");
foreach (string s in differenceQuery)
Console.WriteLine(s);
/* Output:
The following lines are in names1.txt but not names2.txt
Potra, Cristina
Noriega, Fabricio
Aw, Kam Foo
Toyoshima, Tim
Guy, Wey Yuan
Garcia, Debra
*/
Some types of query operations, such as Except, Distinct, Union, and Concat, can only be expressed in method-based syntax.
How to combine and compare string collections
This example shows how to merge files that contain lines of text and then sort the results. Specifically, it shows how to perform a concatenation, a union, and an intersection on the two sets of text lines. It uses the same two text files shows in the preceding example. The code shows examples of the Enumerable.Concat, Enumerable.Union, and Enumerable.Except.
//Put text files in your solution folder
string[] fileA = File.ReadAllLines("names1.txt");
string[] fileB = File.ReadAllLines("names2.txt");
//Simple concatenation and sort. Duplicates are preserved.
var concatQuery = fileA.Concat(fileB).OrderBy(s => s);
// Pass the query variable to another function for execution.
OutputQueryResults(concatQuery, "Simple concatenate and sort. Duplicates are preserved:");
// Concatenate and remove duplicate names based on
// default string comparer.
var uniqueNamesQuery = fileA.Union(fileB).OrderBy(s => s);
OutputQueryResults(uniqueNamesQuery, "Union removes duplicate names:");
// Find the names that occur in both files (based on
// default string comparer).
var commonNamesQuery = fileA.Intersect(fileB);
OutputQueryResults(commonNamesQuery, "Merge based on intersect:");
// Find the matching fields in each list. Merge the two
// results by using Concat, and then
// sort using the default string comparer.
string nameMatch = "Garcia";
var tempQuery1 = from name in fileA
let n = name.Split(',')
where n[0] == nameMatch
select name;
var tempQuery2 = from name2 in fileB
let n2 = name2.Split(',')
where n2[0] == nameMatch
select name2;
var nameMatchQuery = tempQuery1.Concat(tempQuery2).OrderBy(s => s);
OutputQueryResults(nameMatchQuery, $"""Concat based on partial name match "{nameMatch}":""");
static void OutputQueryResults(IEnumerable<string> query, string message)
{
Console.WriteLine(Environment.NewLine + message);
foreach (string item in query)
{
Console.WriteLine(item);
}
Console.WriteLine($"{query.Count()} total names in list");
}
/* Output:
Simple concatenate and sort. Duplicates are preserved:
Aw, Kam Foo
Bankov, Peter
Bankov, Peter
Beebe, Ann
Beebe, Ann
El Yassir, Mehdi
Garcia, Debra
Garcia, Hugo
Garcia, Hugo
Giakoumakis, Leo
Gilchrist, Beth
Guy, Wey Yuan
Holm, Michael
Holm, Michael
Liu, Jinghao
McLin, Nkenge
Myrcha, Jacek
Noriega, Fabricio
Potra, Cristina
Toyoshima, Tim
20 total names in list
Union removes duplicate names:
Aw, Kam Foo
Bankov, Peter
Beebe, Ann
El Yassir, Mehdi
Garcia, Debra
Garcia, Hugo
Giakoumakis, Leo
Gilchrist, Beth
Guy, Wey Yuan
Holm, Michael
Liu, Jinghao
McLin, Nkenge
Myrcha, Jacek
Noriega, Fabricio
Potra, Cristina
Toyoshima, Tim
16 total names in list
Merge based on intersect:
Bankov, Peter
Holm, Michael
Garcia, Hugo
Beebe, Ann
4 total names in list
Concat based on partial name match "Garcia":
Garcia, Debra
Garcia, Hugo
Garcia, Hugo
3 total names in list
*/
How to populate object collections from multiple sources
This example shows how to merge data from different sources into a sequence of new types.
Note
Don't try to join in-memory data or data in the file system with data that is still in a database. Such cross-domain joins can yield undefined results because of different ways in which join operations might be defined for database queries and other types of sources. Additionally, there is a risk that such an operation could cause an out-of-memory exception if the amount of data in the database is large enough. To join data from a database to in-memory data, first call ToList or ToArray on the database query, and then perform the join on the returned collection.
This example uses two files. The first, names.csv, contains student names and student IDs.
Omelchenko,Svetlana,111
O'Donnell,Claire,112
Mortensen,Sven,113
Garcia,Cesar,114
Garcia,Debra,115
Fakhouri,Fadi,116
Feng,Hanying,117
Garcia,Hugo,118
Tucker,Lance,119
Adams,Terry,120
Zabokritski,Eugene,121
Tucker,Michael,122
The second, scores.csv, contains student IDs in the first column, followed by exam scores.
111, 97, 92, 81, 60
112, 75, 84, 91, 39
113, 88, 94, 65, 91
114, 97, 89, 85, 82
115, 35, 72, 91, 70
116, 99, 86, 90, 94
117, 93, 92, 80, 87
118, 92, 90, 83, 78
119, 68, 79, 88, 92
120, 99, 82, 81, 79
121, 96, 85, 91, 60
122, 94, 92, 91, 91
The following example shows how to use a named record Student to store merged data from two in-memory collections of strings that simulate spreadsheet data in .csv format. The ID is used as the key to map students to their scores.
// Each line of names.csv consists of a last name, a first name, and an
// ID number, separated by commas. For example, Omelchenko,Svetlana,111
string[] names = File.ReadAllLines("names.csv");
// Each line of scores.csv consists of an ID number and four test
// scores, separated by commas. For example, 111, 97, 92, 81, 60
string[] scores = File.ReadAllLines("scores.csv");
// Merge the data sources using a named type.
// var could be used instead of an explicit type. Note the dynamic
// creation of a list of ints for the ExamScores member. The first item
// is skipped in the split string because it is the student ID,
// not an exam score.
IEnumerable<Student> queryNamesScores = from nameLine in names
let splitName = nameLine.Split(',')
from scoreLine in scores
let splitScoreLine = scoreLine.Split(',')
where Convert.ToInt32(splitName[2]) == Convert.ToInt32(splitScoreLine[0])
select new Student
(
FirstName: splitName[0],
LastName: splitName[1],
ID: Convert.ToInt32(splitName[2]),
ExamScores: (from scoreAsText in splitScoreLine.Skip(1)
select Convert.ToInt32(scoreAsText)
).ToArray()
);
// Optional. Store the newly created student objects in memory
// for faster access in future queries. This could be useful with
// very large data files.
List<Student> students = queryNamesScores.ToList();
// Display each student's name and exam score average.
foreach (var student in students)
{
Console.WriteLine($"The average score of {student.FirstName} {student.LastName} is {student.ExamScores.Average()}.");
}
/* Output:
The average score of Omelchenko Svetlana is 82.5.
The average score of O'Donnell Claire is 72.25.
The average score of Mortensen Sven is 84.5.
The average score of Garcia Cesar is 88.25.
The average score of Garcia Debra is 67.
The average score of Fakhouri Fadi is 92.25.
The average score of Feng Hanying is 88.
The average score of Garcia Hugo is 85.75.
The average score of Tucker Lance is 81.75.
The average score of Adams Terry is 85.25.
The average score of Zabokritski Eugene is 83.
The average score of Tucker Michael is 92.
*/
In the select clause, each new Student object is initialized from the data in the two sources.
If you don't have to store the results of a query, tuples or anonymous types can be more convenient than named types. The following example executes the same task as the previous example, but uses tuples instead of named types:
// Merge the data sources by using an anonymous type.
// Note the dynamic creation of a list of ints for the
// ExamScores member. We skip 1 because the first string
// in the array is the student ID, not an exam score.
var queryNamesScores2 = from nameLine in names
let splitName = nameLine.Split(',')
from scoreLine in scores
let splitScoreLine = scoreLine.Split(',')
where Convert.ToInt32(splitName[2]) == Convert.ToInt32(splitScoreLine[0])
select (FirstName: splitName[0],
LastName: splitName[1],
ExamScores: (from scoreAsText in splitScoreLine.Skip(1)
select Convert.ToInt32(scoreAsText))
.ToList()
);
// Display each student's name and exam score average.
foreach (var student in queryNamesScores2)
{
Console.WriteLine($"The average score of {student.FirstName} {student.LastName} is {student.ExamScores.Average()}.");
}
How to query an ArrayList with LINQ
When using LINQ to query nongeneric IEnumerable collections such as ArrayList, you must explicitly declare the type of the range variable to reflect the specific type of the objects in the collection. If you have an ArrayList of Student objects, your from clause should look like this:
var query = from Student s in arrList
//...
By specifying the type of the range variable, you're casting each item in the ArrayList to a Student.
The use of an explicitly typed range variable in a query expression is equivalent to calling the Cast method. Cast throws an exception if the specified cast can't be performed. Cast and OfType are the two Standard Query Operator methods that operate on nongeneric IEnumerable types. For more information, see Type Relationships in LINQ Query Operations. The following example shows a query over an ArrayList.
ArrayList arrList = new ArrayList();
arrList.Add(
new Student
(
FirstName: "Svetlana",
LastName: "Omelchenko",
ExamScores: new int[] { 98, 92, 81, 60 }
));
arrList.Add(
new Student
(
FirstName: "Claire",
LastName: "O’Donnell",
ExamScores: new int[] { 75, 84, 91, 39 }
));
arrList.Add(
new Student
(
FirstName: "Sven",
LastName: "Mortensen",
ExamScores: new int[] { 88, 94, 65, 91 }
));
arrList.Add(
new Student
(
FirstName: "Cesar",
LastName: "Garcia",
ExamScores: new int[] { 97, 89, 85, 82 }
));
var query = from Student student in arrList
where student.ExamScores[0] > 95
select student;
foreach (Student s in query)
Console.WriteLine(s.LastName + ": " + s.ExamScores[0]);
